Tools for assessing the scalability of innovations in health: a systematic review.

BACKGROUND: The last decade has seen growing interest in scaling up of innovations to strengthen healthcare systems. However, the lack of appropriate methods for determining their potential for scale-up is an unfortunate global handicap. Thus, we aimed to review tools proposed for assessing the scalability of innovations in health. METHODS: We conducted a systematic review following the COSMIN methodology. We included any empirical research which aimed to investigate the creation, validation or interpretability of a scalability assessment tool in health. We searched Embase, MEDLINE, CINAHL, Web of Science, PsycINFO, Cochrane Library and ERIC from their inception to 20 March 2019. We also searched relevant websites, screened the reference lists of relevant reports and consulted experts in the field. Two reviewers independently selected and extracted eligible reports and assessed the methodological quality of tools. We summarized data using a narrative approach involving thematic syntheses and descriptive statistics. RESULTS: We identified 31 reports describing 21 tools. Types of tools included criteria (47.6%), scales (33.3%) and checklists (19.0%). Most tools were published from 2010 onwards (90.5%), in open-access sources (85.7%) and funded by governmental or nongovernmental organizations (76.2%). All tools were in English; four were translated into French or Spanish (19.0%). Tool creation involved single (23.8%) or multiple (19.0%) types of stakeholders, or stakeholder involvement was not reported (57.1%). No studies reported involving patients or the public, or reported the sex of tool creators. Tools were created for use in high-income countries (28.6%), low- or middle-income countries (19.0%), or both (9.5%), or for transferring innovations from low- or middle-income countries to high-income countries (4.8%). Healthcare levels included public or population health (47.6%), primary healthcare (33.3%) and home care (4.8%). Most tools provided limited information on content validity (85.7%), and none reported on other measurement properties. The methodological quality of tools was deemed inadequate (61.9%) or doubtful (38.1%). CONCLUSIONS: We inventoried tools for assessing the scalability of innovations in health. Existing tools are as yet of limited utility for assessing scalability in health. More work needs to be done to establish key psychometric properties of these tools. Trial registration We registered this review with PROSPERO (identifier: CRD42019107095).

The how and why of producing policy relevant research: perspectives of Australian childhood obesity prevention researchers and policy makers.

BACKGROUND: Understanding why research is conducted may help address the under-utilisation of research. This study examined the reasons for childhood obesity prevention knowledge production in New South Wales (NSW), Australia, and the factors influencing research choices from the perspective of the researchers and health policy agencies contributing to the research. METHODS: A literature search of SCOPUS and ISI Web of Knowledge (affiliation and key word searches) was conducted to compile a database of NSW childhood obesity research outputs, published between 2000 and 2015 (n = 543). Descriptive statistics were used to quantify outputs by research type, differentiating measurement, descriptive, and intervention research, systematic reviews and other publications. Interviews were conducted with a sample of researchers drawn from the database (n = 13) and decision makers from health policy agencies who funded and contributed to childhood obesity research in NSW (n = 15). Researcher interviews examined views about societal impacts, why and under what circumstances the research was conducted. Decision-maker interviews examined policy agency research investment and how research was used in decision making. Content analysis and a thematic approach was used to analyse the interview transcripts. RESULTS: The research in this case was conducted for mix of reasons including those traditionally associated with academic inquiry, as well as intentions to influence policy and practice. Differences in funding mechanisms, administrative and employment arrangements, and 'who' initiated the research, created differing incentives and perspectives for knowledge production. Factors associated with the characteristics and experience of the individuals involved also influenced goals, as did the type of research conducted. Policy agencies played a role in directing research to address policy needs. CONCLUSIONS: The findings of this study confirm that researchers are strongly influenced by their working environment. Funding schemes and other incentives to support policy relevant knowledge production are important. Contextual factors such as policy priorities, policy-driven research funding and the embedded nature or strong connections between some researchers and the policy agencies involved, are likely to have influenced the extent to which policy goals were reported in this study.

Identification and evaluation of risk of generalizability biases in pilot versus efficacy/effectiveness trials: a systematic review and meta-analysis.

BACKGROUND: Preliminary evaluations of behavioral interventions, referred to as pilot studies, predate the conduct of many large-scale efficacy/effectiveness trial. The ability of a pilot study to inform an efficacy/effectiveness trial relies on careful considerations in the design, delivery, and interpretation of the pilot results to avoid exaggerated early discoveries that may lead to subsequent failed efficacy/effectiveness trials. "Risk of generalizability biases (RGB)" in pilot studies may reduce the probability of replicating results in a larger efficacy/effectiveness trial. We aimed to generate an operational list of potential RGBs and to evaluate their impact in pairs of published pilot studies and larger, more well-powered trial on the topic of childhood obesity. METHODS: We conducted a systematic literature review to identify published pilot studies that had a published larger-scale trial of the same or similar intervention. Searches were updated and completed through December 31st, 2018. Eligible studies were behavioral interventions involving youth (≤18 yrs) on a topic related to childhood obesity (e.g., prevention/treatment, weight reduction, physical activity, diet, sleep, screen time/sedentary behavior). Extracted information included study characteristics and all outcomes. A list of 9 RGBs were defined and coded: intervention intensity bias, implementation support bias, delivery agent bias, target audience bias, duration bias, setting bias, measurement bias, directional conclusion bias, and outcome bias. Three reviewers independently coded for the presence of RGBs. Multi-level random effects meta-analyses were performed to investigate the association of the biases to study outcomes. RESULTS: A total of 39 pilot and larger trial pairs were identified. The frequency of the biases varied: delivery agent bias (19/39 pairs), duration bias (15/39), implementation support bias (13/39), outcome bias (6/39), measurement bias (4/39), directional conclusion bias (3/39), target audience bias (3/39), intervention intensity bias (1/39), and setting bias (0/39). In meta-analyses, delivery agent, implementation support, duration, and measurement bias were associated with an attenuation of the effect size of - 0.325 (95CI - 0.556 to - 0.094), - 0.346 (- 0.640 to - 0.052), - 0.342 (- 0.498 to - 0.187), and - 0.360 (- 0.631 to - 0.089), respectively. CONCLUSIONS: Pre-emptive avoidance of RGBs during the initial testing of an intervention may diminish the voltage drop between pilot and larger efficacy/effectiveness trials and enhance the odds of successful translation.

What is generated and what is used: a description of public health research output and citation.

The aim of this short report was to describe the output and citation rates of public health. Data-based publications and literature reviews from the year 2008, and their 5-year citation rates were extracted from 10 randomly selected public health journals. In total, 86.2% of publications were descriptive/epidemiological studies, 56.8% used cross-sectional (56.8%) designs and 77.8% were classified as research translation stage 2. Reviews and publications describing randomized controlled trials were the most highly cited, but were infrequently published. Strategies to address the discordance between public health research output and research citation may improve the impact of public health research.

Tracking funded health intervention research.

OBJECTIVE: To describe the research publication outputs from intervention research funded by Australia's National Health and Medical Research Council (NHMRC). DESIGN AND SETTING: Analysis of descriptive data and data on publication outputs collected between 23 July 2012 and 10 December 2013 relating to health intervention research project grants funded between 1 January 2003 and 31 December 2007. MAIN OUTCOME MEASURES: Stages of development of intervention studies (efficacy, effectiveness, replication, adaptation or dissemination of intervention); types of interventions studied; publication output per NHMRC grant; and whether interventions produced statistically significant changes in primary outcome variables. RESULTS: Most of the identified studies tested intervention efficacy or effectiveness in clinical or community settings, with few testing the later stages of intervention development, such as replication, adaptation or dissemination. Studies focused largely on chronic disease treatment and management, and encompassed various medical and allied health disciplines. Equal numbers of studies had interventions that produced statistically significant results on primary outcomes, (27) and those that did not (27). The mean number of total published articles per grant was 3.3, with 2.0 articles per grant focusing on results, and the remainder covering descriptive, exploratory or methodological aspects of intervention research. CONCLUSIONS: Our study provides a benchmark for the publication outputs of NHMRC-funded health intervention research in Australia. Research productivity is particularly important for intervention research, where findings are likely to have more immediate and direct applicability to health policy and practice. Tracking research outputs in this way provides information on whether current research investment patterns match the need for evidence about health care interventions.

A narrative review of research impact assessment models and methods.

BACKGROUND: Research funding agencies continue to grapple with assessing research impact. Theoretical frameworks are useful tools for describing and understanding research impact. The purpose of this narrative literature review was to synthesize evidence that describes processes and conceptual models for assessing policy and practice impacts of public health research. METHODS: The review involved keyword searches of electronic databases, including MEDLINE, CINAHL, PsycINFO, EBM Reviews, and Google Scholar in July/August 2013. Review search terms included 'research impact', 'policy and practice', 'intervention research', 'translational research', 'health promotion', and 'public health'. The review included theoretical and opinion pieces, case studies, descriptive studies, frameworks and systematic reviews describing processes, and conceptual models for assessing research impact. The review was conducted in two phases: initially, abstracts were retrieved and assessed against the review criteria followed by the retrieval and assessment of full papers against review criteria. RESULTS: Thirty one primary studies and one systematic review met the review criteria, with 88% of studies published since 2006. Studies comprised assessments of the impacts of a wide range of health-related research, including basic and biomedical research, clinical trials, health service research, as well as public health research. Six studies had an explicit focus on assessing impacts of health promotion or public health research and one had a specific focus on intervention research impact assessment. A total of 16 different impact assessment models were identified, with the 'payback model' the most frequently used conceptual framework. Typically, impacts were assessed across multiple dimensions using mixed methodologies, including publication and citation analysis, interviews with principal investigators, peer assessment, case studies, and document analysis. The vast majority of studies relied on principal investigator interviews and/or peer review to assess impacts, instead of interviewing policymakers and end-users of research. CONCLUSIONS: Research impact assessment is a new field of scientific endeavour and there are a growing number of conceptual frameworks applied to assess the impacts of research.

Does health intervention research have real world policy and practice impacts: testing a new impact assessment tool.

BACKGROUND: There is a growing emphasis on the importance of research having demonstrable public benefit. Measurements of the impacts of research are therefore needed. We applied a modified impact assessment process that builds on best practice to 5 years (2003-2007) of intervention research funded by Australia's National Health and Medical Research Council to determine if these studies had post-research real-world policy and practice impacts. METHODS: We used a mixed method sequential methodology whereby chief investigators of eligible intervention studies who completed two surveys and an interview were included in our final sample (n = 50), on which we conducted post-research impact assessments. Data from the surveys and interviews were triangulated with additional information obtained from documentary analysis to develop comprehensive case studies. These case studies were then summarized and the reported impacts were scored by an expert panel using criteria for four impact dimensions: corroboration; attribution, reach, and importance. RESULTS: Nineteen (38%) of the cases in our final sample were found to have had policy and practice impacts, with an even distribution of high, medium, and low impact scores. While the tool facilitated a rigorous and explicit criterion-based assessment of post-research impacts, it was not always possible to obtain evidence using documentary analysis to corroborate the impacts reported in chief investigator interviews. CONCLUSIONS: While policy and practice is ideally informed by reviews of evidence, some intervention research can and does have real world impacts that can be attributed to single studies. We recommend impact assessments apply explicit criteria to consider the corroboration, attribution, reach, and importance of reported impacts on policy and practice. Impact assessments should also allow sufficient time between impact data collection and completion of the original research and include mechanisms to obtain end-user input to corroborate claims and reduce biases that result from seeking information from researchers only.

How can knowledge exchange portals assist in knowledge management for evidence-informed decision making in public health?

BACKGROUND: Knowledge exchange portals are emerging as web tools that can help facilitate knowledge management in public health. We conducted a review to better understand the nature of these portals and their contribution to knowledge management in public health, with the aim of informing future development of portals in this field. METHODS: A systematic literature search was conducted of the peer-reviewed and grey literature to identify articles that described the design, development or evaluation of Knowledge Exchange Portals KEPs in the public health field. The content of the articles was analysed, interpreted and synthesised in light of the objectives of the review. RESULTS: The systematic search yielded 2223 articles, of which fifteen were deemed eligible for review, including eight case studies, six evaluation studies and one commentary article. Knowledge exchange portals mainly included design features to support knowledge access and creation, but formative evaluation studies examining user needs suggested collaborative features supporting knowledge exchange would also be useful. Overall web usage statistics revealed increasing use of some of these portals over time; however difficulties remain in retaining users. There is some evidence to suggest that the use of a knowledge exchange portal in combination with tailored and targeted messaging can increase the use of evidence in policy and program decision making at the organisational level. CONCLUSIONS: Knowledge exchange portals can be a platform for providing integrated access to relevant content and resources in one location, for sharing and distributing information and for bringing people together for knowledge exchange. However more performance evaluation studies are needed to determine how they can best support evidence-informed decision making in public health.

Increasing the scale and adoption of population health interventions: experiences and perspectives of policy makers, practitioners, and researchers.

BACKGROUND: Decisions to scale up population health interventions from small projects to wider state or national implementation is fundamental to maximising population-wide health improvements. The objectives of this study were to examine: i) how decisions to scale up interventions are currently made in practice; ii) the role that evidence plays in informing decisions to scale up interventions; and iii) the role policy makers, practitioners, and researchers play in this process. METHODS: Interviews with an expert panel of senior Australian and international public health policy-makers (n = 7), practitioners (n = 7), and researchers (n = 7) were conducted in May 2013 with a participation rate of 84%. RESULTS: Scaling up decisions were generally made through iterative processes and led by policy makers and/or practitioners, but ultimately approved by political leaders and/or senior executives of funding agencies. Research evidence formed a component of the overall set of information used in decision-making, but its contribution was limited by the paucity of relevant intervention effectiveness research, and data on costs and cost effectiveness. Policy makers, practitioners/service managers, and researchers had different, but complementary roles to play in the process of scaling up interventions. CONCLUSIONS: This analysis articulates the processes of how decisions to scale up interventions are made, the roles of evidence, and contribution of different professional groups. More intervention research that includes data on the effectiveness, reach, and costs of operating at scale and key service delivery issues (including acceptability and fit of interventions and delivery models) should be sought as this has the potential to substantially advance the relevance and ultimately usability of research evidence for scaling up population health action.

'Translational formative evaluation': critical in up-scaling public health programmes.

The process of generating evidence-based public health interventions is understood to include steps that define the issue, generate and test solutions in controlled settings, replicate and then disseminate more widely. However, to date models have not considered the types and scale of formative evaluation tasks that are needed to up-scale interventions, from efficacy to population-wide dissemination in the real world. In this paper, we propose that an additional stage of 'translational formative evaluation' is necessary for the translation of effectiveness evidence into wide-scale public health practice. We illustrate the utility of translational formative evaluation, through a case study of the Get Healthy Information and Coaching Service(®) (GHS), a population-based telephone service designed to assist adults change lifestyle-related behaviours. The additional translational formative evaluation steps comprised synthesis of efficacy studies, qualitative research with the wider target audience, environmental analysis and stakeholder consultation. They produced precise recommendations to refine GHS design and implementation. Translational formative evaluation is a necessary intermediate step, following efficacy studies and a precursor to population-wide implementation of public health programmes.

Harnessing the power of advertising to prevent childhood obesity.

BACKGROUND: Social marketing integrates communication campaigns with behavioural and environmental change strategies. Childhood obesity programs could benefit significantly from social marketing but communication campaigns on this issue tend to be stand-alone. METHODS: A large-scale multi-setting child obesity prevention program was implemented in the Hunter New England (HNE) region of New South Wales (NSW), Australia from 2005-2010. The program included a series of communication campaigns promoting the program and its key messages: drinking water; getting physically active and; eating more vegetables and fruit. Pre-post telephone surveys (n = 9) were undertaken to evaluate awareness of the campaigns among parents of children aged 2-15 years using repeat cross-sections of randomly selected cohorts. A total of 1,367 parents (HNE = 748, NSW = 619) participated. RESULTS: At each survey post baseline, HNE parents were significantly more likely to have seen, read or heard about the program and its messages in the media than parents in the remainder of the state (p < 0.001). Further, there was a significant increase in awareness of the program and each of its messages over time in HNE compared to no change over time in NSW (p < 0.001). Awareness was significantly higher (p < 0.05) in HNE compared to NSW after each specific campaign (except the vegetable one) and significantly higher awareness levels were sustained for each campaign until the end of the program. At the end of the program participants without a tertiary education were significantly more likely (p = 0.04) to be aware of the brand campaign (31%) than those with (20%) but there were no other statistically significant socio-demographic differences in awareness. CONCLUSIONS: The Good for Kids communication campaigns increased and maintained awareness of childhood obesity prevention messages. Moreover, messages were delivered equitably to diverse socio-demographic groups within the region.

Policy and practice impacts of applied research: a case study analysis of the New South Wales Health Promotion Demonstration Research Grants Scheme 2000-2006.

BACKGROUND: Intervention research provides important information regarding feasible and effective interventions for health policy makers, but few empirical studies have explored the mechanisms by which these studies influence policy and practice. This study provides an exploratory case series analysis of the policy, practice and other related impacts of the 15 research projects funded through the New South Wales Health Promotion Demonstration Research Grants Scheme during the period 2000 to 2006, and explored the factors mediating impacts. METHODS: Data collection included semi-structured interviews with the chief investigators (n = 17) and end-users (n = 29) of each of the 15 projects to explore if, how and under what circumstances the findings had been used, as well as bibliometric analysis and verification using documentary evidence. Data analysis involved thematic coding of interview data and triangulation with other data sources to produce case summaries of impacts for each project. Case summaries were then individually assessed against four impact criteria and discussed at a verification panel meeting where final group assessments of the impact of research projects were made and key influences of research impact identified. RESULTS: Funded projects had variable impacts on policy and practice. Project findings were used for agenda setting (raising awareness of issues), identifying areas and target groups for interventions, informing new policies, and supporting and justifying existing policies and programs across sectors. Reported factors influencing the use of findings were: i) nature of the intervention; ii) leadership and champions; iii) research quality; iv) effective partnerships; v) dissemination strategies used; and, vi) contextual factors. CONCLUSIONS: The case series analysis provides new insights into how and under what circumstances intervention research is used to influence real world policy and practice. The findings highlight that intervention research projects can achieve the greatest policy and practice impacts if they address proximal needs of the policy context by engaging end-users from the inception of projects and utilizing existing policy networks and structures, and using a range of strategies to disseminate findings that go beyond traditional peer review publications.

Evaluation framework for translational research: case study of Australia's get healthy information and coaching service(R).

The Get Healthy Information and Coaching Service® (GHS), a free government-funded telephone-delivered information and coaching service was launched in February 2009 by the Australian New South Wales state government. It represents the translation of research evidence applied in the real world (T4 or Phase 4 translation), aimed at addressing the modifiable risk factors associated with the overweight and obesity. In controlled settings, it has been established that telephone-based lifestyle counseling programs are efficacious in reducing anthropometric and behavioral risk factors. This article presents the GHS case study as a population-wide intervention and describes the quasi-experimental evaluation framework used to evaluate both the process (statewide implementation) and impact (effectiveness) of the GHS in a real-world environment. It details the data collection, measures, and statistical analysis required in assessing the process of implementation-reach and recruitment, marketing and promotion, service satisfaction, intervention fidelity, and GHS setting up and operations costs-and in assessing the impact of GHS-increasing physical activity, improving dietary practices, and reducing body weight and waist circumference. The comprehensive evaluation framework designed for the GHS provides a method for building effectiveness evidence of a rare translation of efficacy trial evidence into population-wide practice.

Impact of a population based intervention to increase the adoption of multiple physical activity practices in centre based childcare services: a quasi experimental, effectiveness study.

BACKGROUND: There is considerable scope to improve the delivery of practices that increase the physical activity of children in centre based childcare services. Few studies have reported the effectiveness of interventions to address this, particularly at a population level. The primary aim of this study was to describe the impact of an intervention to increase the adoption of multiple policies and practices to promote physical activity in centre based childcare services. METHODS: A quasi experimental study was conducted in centre based childcare services (n =228) in New South Wales (NSW), Australia and involved a three month intervention to increase the adoption of eight practices within childcare services that have been suggested to promote child physical activity. Intervention strategies to support the adoption of practices included staff training, resources, incentives, follow-up support and performance monitoring and feedback. Randomly selected childcare services in the remainder of NSW acted as a comparison group (n = 164) and did not receive the intervention but may have been exposed to a concurrent NSW government healthy eating and physical activity initiative. Self reported information on physical activity policies, fundamental movement skills sessions, structured physical activity opportunities, staff involvement in active play and provision of verbal prompts to encourage physical activity, small screen recreation opportunities, sedentary time, and staff trained in physical activity were collected by telephone survey with childcare service managers at baseline and 18 months later. RESULTS: Compared with the comparison area, the study found significantly greater increases in the prevalence of intervention services with a written physical activity policy, with policy referring to placing limits on small screen recreation, and with staff trained in physical activity. In addition, non-significant trends towards a greater increase in the proportion of intervention services conducting daily fundamental movement skill sessions, and such services having a physical activity policy supporting physical activity training for staff were also evident. CONCLUSION: The intervention was effective in improving a number of centre based childcare service policies and practices associated with promoting child physical activity. Adoption of a broader range of practices may require more intensive and prolonged intervention support.

Effectiveness of a multi-strategy intervention in increasing the implementation of vegetable and fruit breaks by Australian primary schools: a non-randomized controlled trial.

BACKGROUND: Limited evidence exists describing the effectiveness of strategies in facilitating the implementation of vegetable and fruit programs by schools on a population wide basis. The aim of this study was to examine the effectiveness of a multi-strategy intervention in increasing the population-wide implementation of vegetable and fruit breaks by primary schools and to determine if intervention effectiveness varied by school characteristics. METHODS: A quasi-experimental study was conducted in primary schools in the state of New South Wales, Australia. All primary schools in one region of the state (n = 422) received a multi-strategy intervention. A random sample of schools (n = 406) in the remainder of the state served as comparison schools. The multi-strategy intervention to increase vegetable and fruit breaks involved the development and provision of: program consensus and leadership; staff training; program materials; incentives; follow-up support; and implementation feedback. Comparison schools had access to routine information-based Government support. Data to assess the prevalence of vegetable and fruit breaks were collected by telephone from Principals of the intervention and comparison schools at baseline (2006-2007) and 11 to 15 months following the commencement of the intervention (2009-2010). GEE analysis was used to examine the change in the prevalence of vegetable and fruit breaks in intervention schools compared to comparison schools. RESULTS: At follow-up, prevalence of vegetable and fruit breaks increased significantly in both intervention (50.3% to 82.0%, p < 0.001) and comparison (45.4% to 60.9% p < 0.001) schools. The increase in prevalence in intervention schools was significantly larger than among comparison schools (OR 2.36; 95% CI 1.60-3.49, p <0.001). The effect size was similar between schools regardless of the rurality or socioeconomic status of school location, school size or government or non-government school type. CONCLUSION: The findings suggest that a multi-strategy intervention can significantly increase the implementation of vegetable and fruit breaks by a large number of Australian primary schools.

Research to practice: application of an evidence-building framework to a childhood obesity prevention initiative in New South Wales.

ISSUE ADDRESSED: Building evidence-based health promotion programs involves a number of steps. This paper aims to develop a set of criteria for assessing the evidence available according to a five-stage evidence-building framework, and apply these criteria to current child obesity prevention programs in NSW to determine the usefulness of the framework in identifying gaps in evidence and opportunities for future research and evaluation. METHODS: A set of scoring criteria were developed for application within the five stages of an 'evidence-building' framework: problem definition, solution generation, intervention testing (efficacy), intervention replication, and dissemination research. The research evidence surrounding the 10 childhood obesity prevention programs planned for state-wide implementation in the New South Wales Healthy Children Initiative (HCI) was identified and examined using these criteria within the framework. RESULTS: The evidence for the component programs of the HCI is at different stages of development. While problem definition and, to a lesser extent, solution generation was thoroughly addressed across all programs, there were a number of evidence gaps, indicating research opportunities for efficacy testing and intervention replication across a variety of settings and populations. CONCLUSIONS: The five-stage evidence-building framework helped identify important research and evaluation opportunities that could improve health promotion practice in NSW. More work is needed to determine the validity and reliability of the criteria for rating the extent and quality of the evidence for each stage.

The MOHMQuit (Midwives and Obstetricians Helping Mothers to Quit Smoking) Trial: protocol for a stepped-wedge implementation trial to improve best practice smoking cessation support in public antenatal care services.

BACKGROUND: Smoking during pregnancy is the most important preventable cause of adverse pregnancy outcomes, yet smoking cessation support (SCS) is inconsistently provided. The MOMHQUIT intervention was developed to address this evidence-practice gap, using the Behaviour Change Wheel method by mapping barriers to intervention strategies. MOHMQuit includes systems, leadership and clinician elements. This implementation trial will determine the effectiveness and cost-effectiveness of MOHMQuit in improving smoking cessation rates in pregnant women in public maternity care services in Australia; test the mechanisms of action of the intervention strategies; and examine implementation outcomes. METHODS: A stepped-wedge cluster-randomised design will be used. Implementation of MOHMQuit will include reinforcing leadership investment in SCS as a clinical priority, strengthening maternity care clinicians' knowledge, skills, confidence and attitudes towards the provision of SCS, and clinicians' documentation of guideline-recommended SCS provided during antenatal care. Approximately, 4000 women who report smoking during pregnancy will be recruited across nine sites. The intervention and its implementation will be evaluated using a mixed methods approach. The primary outcome will be 7-day point prevalence abstinence at the end of pregnancy, among pregnant smokers, verified by salivary cotinine testing. Continuous data collection from electronic medical records and telephone interviews with postpartum women will occur throughout 32 months of the trial to assess changes in cessation rates reported by women, and SCS documented by clinicians and reported by women. Data collection to assess changes in clinicians' knowledge, skills, confidence and attitudes will occur prior to and immediately after the intervention at each site, and again 6 months later. Questionnaires at 3 months following the intervention, and semi-structured interviews at 6 months with maternity service leaders will explore leaders' perceptions of acceptability, adoption, appropriateness, feasibility, adaptations and fidelity of delivery of the MOHMQuit intervention. Structural equation modelling will examine causal linkages between the strategies, mediators and outcomes. Cost-effectiveness analyses will also be undertaken. DISCUSSION: This study will provide evidence of the effectiveness of a multi-level implementation intervention to support policy decisions; and evidence regarding mechanisms of action of the intervention strategies (how the strategies effected outcomes) to support further theoretical developments in implementation science. TRIAL REGISTRATION: ACTRN12622000167763, registered February 2nd 2022.

Tensions and Paradoxes of Scaling Up: A Critical Reflection on Physical Activity Promotion.

BACKGROUND: Achieving system-level, sustainable 'scale-up' of interventions is the epitome of successful translation of evidence-based approaches in population health. In physical activity promotion, few evidence-based interventions reach implementation at scale or become embedded within systems for sustainable health impact. This is despite the vast published literature describing efficacy studies of small-scale physical activity interventions. Research into physical activity scale-up (through case-study analysis; evaluations of scale-up processes in implementation trials; and mapping the processes, strategies, and principles for scale-up) has identified barriers and facilitators to intervention expansion. Many interventions are implemented at scale by governments but have not been evaluated or have unpublished evaluation information. Further, few public health interventions have evaluations that reveal the costs and benefits of scaled-up implementation. This lack of economic information introduces an additional element of risk for decision makers when deciding which physical activity interventions should be supported with scarce funding resources. Decision-makers face many other challenges when scaling interventions which do not relate to formal research trials of scale-up; Methods: To explore these issues, a multidisciplinary two-day workshop involving experts in physical activity scale-up was convened by the University of Newcastle, Australia, and the University of Ottawa, Canada (February 2019); Results: In this paper we discuss some of the scale-up tensions (challenges and conflicts) and paradoxes (things that are contrary to expectations) that emerged from this workshop in the context of the current literature and our own experiences in this field. We frame scale-up tensions according to epistemology, methodology, time, and partnerships; and paradoxes as 'reach without scale', 'planned serendipity' and 'simple complexity'. We reflect on the implications of these scale-up tensions and paradoxes, providing considerations for future scale-up research and practice moving forward; Conclusions: In this paper, we delve deeper into stakeholders' assumptions, processes and expectations of scaling up, and challenge in what ways as stakeholders, we all contribute to desired or undesired outcomes. Through a lens of 'tensions' and 'paradoxes', we make an original contribution to the scale-up literature that might influence current perspectives of scaling-up, provide future approaches for physical activity promotion, and contribute to understanding of dynamic of research-practice partnerships.

A systematic review of adaptations and effectiveness of scaled-up nutrition interventions.

CONTEXT: Public health nutrition interventions shown to be effective under optimal research conditions need to be scaled up and implemented in real-world settings. OBJECTIVES: The primary aim for this review was to assess the effectiveness of scaled-up public health nutrition interventions with proven efficacy, as examined in a randomized controlled trial. Secondary objectives were to: 1) determine if the effect size of scaled-up interventions were comparable to the prescale effect, and; 2) identify any adaptations made during the scale-up process. DATA SOURCES: Six electronic databases were searched and field experts contacted. STUDY SELECTION: An intervention was considered scaled up if it was delivered on a larger scale than a preceding randomized controlled trial ("prescale") in which a significant intervention effect (P ≤ 0.05) was reported on a measure of nutrition. DATA EXTRACTION: Two reviewers independently performed screening and data extraction. Effect size differences between prescale and scaled-up interventions were quantified. Adaptations to scale-up studies were coded according to the Adaptome model. RESULTS: Ten scaled-up nutrition interventions were identified. The effect size difference between prescale trials and scaled-up studies ranged from -32.2% to 222% (median, 50%). All studies made adaptations between prescale to scaled-up interventions. CONCLUSION: The effects of nutrition interventions implemented at scale typically were half that achieved in prior efficacy trials. Identifying effective scale-up strategies and methods to support retainment of the original prescale effect size is urgently needed to inform public health policy. SYSTEMATIC REVIEW REGISTRATION: PROSPERO registration no.CRD42020149267.

Public health research outputs from efficacy to dissemination: a bibliometric analysis.

BACKGROUND: More intervention research is needed, particularly 'real world' intervention replication and dissemination studies, to optimize improvements in health. This study assessed the proportion and type of published public health intervention research papers over time in physical activity and falls prevention, both important contributors to preventable morbidity and mortality. METHODS: A keyword search was conducted, using Medline and PsycINFO to locate publications in 1988-1989, 1998-1999, and 2008-2009 for the two topic areas. In stage 1, a random sample of 1200 publications per time period for both topics were categorized as: non-public health, non-data-based public health, or data-based public health. In stage 2 data-based public health articles were further classified as measurement, descriptive, etiological or intervention research. Finally, intervention papers were categorized as: efficacy, intervention replication or dissemination studies. Inter-rater reliability of paper classification was 88%. RESULTS: Descriptive studies were the most common data-based papers across all time periods (1988-89; 1998-1999;2008-2009) for both issues (physical activity: 47%; 54%; 65% and falls 75%; 64%; 63%), increasing significantly over time for physical activity. The proportion of intervention publications did not increase over time for physical activity comprising 23% across all time periods and fluctuated for falls across the time periods (10%; 21%; 17%). The proportion of intervention articles that were replication studies increased over the three time periods for physical activity (0%; 2%; 11%) and for falls (0%; 22%; 35%). Dissemination studies first appeared in the literature in 2008-2009, making up only 3% of physical activity and 7% of falls intervention studies. CONCLUSIONS: Intervention research studies remain only a modest proportion of all published studies in physical activity and falls prevention; the majority of the intervention studies, are efficacy studies although there is growing evidence of a move towards replication and dissemination studies, which may have greater potential for improving population health.