Strategies for involving patients and the public in scaling initiatives in health and social services: A scoping review.

BACKGROUND: Scaling in health and social services (HSS) aims to increase the intended impact of proven effective interventions. Patient and public involvement (PPI) is critical for ensuring that scaling beneficiaries' interests are served. We aimed to identify PPI strategies and their characteristics in the science and practice of scaling in HSS. METHODS: In this scoping review, we included any scaling initiative in HSS that used PPI strategies and reported PPI methods and outcomes. We searched electronic databases (e.g., Medline) from inception to 5 February 2024, and grey literature (e.g., Google). Paired reviewers independently selected and extracted eligible reports. A narrative synthesis was performed and we used the PRISMA for Scoping Reviews and the Guidance for Reporting Involvement of Patients and the Public (GRIPP2). FINDINGS: We included 110 unique reports out of 24,579 records. In the past 5 years, the evidence on PPI in scaling has increased faster than in any previous period. We found 236 mutually nonexclusive PPI strategies among 120 scaling initiatives. Twenty-four initiatives did not target a specific country; but most of those that did so (n = 96) occurred in higher-income countries (n = 51). Community-based primary health care was the most frequent level of care (n = 103). Mostly, patients and the public were involved throughout all scaling phases (n = 46) and throughout the continuum of collaboration (n = 45); the most frequently reported ethical lens regarding the rationale for PPI was consequentialist-utilitarian (n = 96). Few papers reported PPI recruitment processes (n = 31) or incentives used (n = 18). PPI strategies occurred mostly in direct care (n = 88). Patient and public education was the PPI strategy most reported (n = 31), followed by population consultations (n = 30). CONCLUSIONS: PPI in scaling is increasing in HSS. Further investigation is needed to better document the PPI experience in scaling and ensure that it occurs in a meaningful and equitable way. PATIENT AND PUBLIC CONTRIBUTION: Two patients were involved in this review. They shared decisions on review questions, data collection instruments, protocol design, and findings dissemination. REVIEW REGISTRATION: Open Science Framework on 19 August 2020 (https://osf.io/zqpx7/).

Research outcomes informing the selection of public health interventions and strategies to implement them: A cross-sectional survey of Australian policy-maker and practitioner preferences.

BACKGROUND: A key role of public health policy-makers and practitioners is to ensure beneficial interventions are implemented effectively enough to yield improvements in public health. The use of evidence to guide public health decision-making to achieve this is recommended. However, few studies have examined the relative value, as reported by policy-makers and practitioners, of different broad research outcomes (that is, measures of cost, acceptability, and effectiveness). To guide the conduct of research and better inform public health policy and practice, this study aimed at describing the research outcomes that Australian policy-makers and practitioners consider important for their decision-making when selecting: (a) public health interventions; (b) strategies to support their implementation; and (c) to assess the differences in research outcome preferences between policy-makers and practitioners. METHOD: An online value-weighting survey was conducted with Australian public health policy-makers and practitioners working in the field of non-communicable disease prevention. Participants were presented with a list of research outcomes and were asked to select up to five they considered most critical to their decision-making. They then allocated 100 points across these - allocating more points to outcomes perceived as more important. Outcome lists were derived from a review and consolidation of evaluation and outcome frameworks in the fields of public health knowledge translation and implementation. We used descriptive statistics to report relative preferences overall and for policy-makers and practitioners separately. RESULTS: Of the 186 participants; 90 primarily identified as policy-makers and 96 as public health prevention practitioners. Overall, research outcomes of effectiveness, equity, feasibility, and sustainability were identified as the four most important outcomes when considering either interventions or strategies to implement them. Scores were similar for most outcomes between policy-makers and practitioners. CONCLUSION: For Australian policy-makers and practitioners working in the field of non-communicable disease prevention, outcomes related to effectiveness, equity, feasibility, and sustainability appear particularly important to their decisions about the interventions they select and the strategies they employ to implement them. The findings suggest researchers should seek to meet these information needs and prioritize the inclusion of such outcomes in their research and dissemination activities. The extent to which these outcomes are critical to informing the decision of policy-makers and practitioners working in other jurisdictions or contexts warrants further investigation.

Evidence on Scaling in Health and Social Care: An Umbrella Review.

Policy Points More rigorous methodologies and systematic approaches should be encouraged in the science of scaling. This will help researchers better determine the effectiveness of scaling, guide stakeholders in the scaling process, and ultimately increase the impacts of health innovations. The practice and the science of scaling need to expand worldwide to address complex health conditions such as noncommunicable and chronic diseases. Although most of the scaling experiences described in the literature are occurring in the Global South, most of the authors publishing on it are based in the Global North. As the science of scaling spreads across the world with the aim of reducing health inequities, it is also essential to address the power imbalance in how we do scaling research globally. CONTEXT: Scaling of effective innovations in health and social care is essential to increase their impact. We aimed to synthesize the evidence base on scaling and identify current knowledge gaps. METHODS: We conducted an umbrella review according to the Joanna Briggs Institute Reviewers' Manual. We included any type of review that 1) focused on scaling, 2) covered health or social care, and 3) presented a methods section. We searched MEDLINE (Ovid), Embase, PsycINFO (Ovid), CINAHL (EBSCO), Web of Science, The Cochrane Library, Sociological Abstracts (ProQuest), Academic Search Premier (EBSCO), and ProQuest Dissertations & Theses Global from their inception to August 6, 2020. We searched the gray literature using, e.g., Google and WHO-ExpandNet. We assessed methodological quality with AMSTAR2. Paired reviewers independently selected and extracted eligible reviews and assessed study quality. A narrative synthesis was performed. FINDINGS: Of 24,269 records, 137 unique reviews were included. The quality of the 58 systematic reviews was critically low (n = 42). The most frequent review type was systematic review (n = 58). Most reported on scaling in low- and middle-income countries (n = 59), whereas most first authors were from high-income countries (n = 114). Most reviews concerned infectious diseases (n = 36) or maternal-child health (n = 28). They mainly focused on interventions (n = 37), barriers and facilitators (n = 29), frameworks (n = 24), scalability (n = 24), and costs (n = 14). The WHO/ExpandNet scaling definition was the definition most frequently used (n = 26). Domains most reported as influencing scaling success were building scaling infrastructure (e.g., creating new service sites) and human resources (e.g., training community health care providers). CONCLUSIONS: The evidence base on scaling is evolving rapidly as reflected by publication trends, the range of focus areas, and diversity of scaling definitions. Our study highlights knowledge gaps around methodology and research infrastructures to facilitate equitable North-South research relationships. Common efforts are needed to ensure scaling expands the impacts of health and social innovations to broader populations.

Study protocol of an economic evaluation embedded in the Midwives and Obstetricians Helping Mothers to Quit Smoking (MOHMQuit) trial.

BACKGROUND: Tobacco smoking during pregnancy is the most important preventable risk factor for pregnancy complications and adverse birth outcomes and can have lifelong consequences for infants. Smoking during pregnancy is associated with higher healthcare costs related to birth complications and during childhood. Psychosocial interventions to support pregnant women to quit are effective, yet provision of smoking cessation support has been inconsistent. The Midwives and Obstetricians Helping Mothers to Quit Smoking (MOHMQuit) intervention provides systems change, and leadership and clinician elements, to support clinicians to help women stop smoking in pregnancy. There have been few long-term analyses conducted of the cost-effectiveness of smoking cessation interventions for pregnant women that target healthcare providers. This protocol describes the economic evaluation of the MOHMQuit trial, a pragmatic stepped-wedge cluster-randomised controlled implementation trial in nine public maternity services in New South Wales (NSW), Australia, to ascertain whether MOHMQuit is cost-effective in supporting clinicians to help women quit smoking in pregnancy compared to usual care. METHODS: Two primary analyses will be carried out comparing MOHMQuit with usual care from an Australian health care system perspective: i) a within-trial cost-effectiveness analysis with results presented as the incremental cost per additional quitter; and ii) a lifetime cost-utility analysis using a published probabilistic decision analytic Markov model with results presented as incremental cost per quality-adjusted life-year (QALY) gained for mother and child. Patient-level data on resource use and outcomes will be used in the within-trial analysis and extrapolated and supplemented with national population statistics and published data from the literature for the lifetime analysis. DISCUSSION: There is increasing demand for information on the cost-effectiveness of implementing healthcare interventions to provide policy makers with critical information for the best value for money within finite budgets. Economic evaluation of the MOHMQuit trial will provide essential, policy-relevant information for decision makers on the value of evidence-based implementation of support for healthcare providers delivering services for pregnant women. TRIAL REGISTRATIONS: ACTRN12622000167763, registered 2 February 2022.

A mixed methods evaluation of Quit for new life, a smoking cessation initiative for women having an Aboriginal baby.

BACKGROUND: Quit for new life (QFNL) is a smoking cessation initiative developed to support mothers of Aboriginal babies to quit smoking during pregnancy. The state-wide initiative provides support for pregnant women and their households including free nicotine replacement therapy (NRT) and follow up cessation advice. Services are also supported to implement systems-level changes and integrate QFNL into routine care. This study aimed to evaluate: (1) models of implementation of QFNL; (2) the uptake of QFNL; (3) the impact of QFNL on smoking behaviours; and (4) stakeholder perceptions of the initiative. METHODS: A mixed methods study was conducted comprising semi-structured interviews and analysis of routinely collected data. Interviews were conducted with 6 clients and 35 stakeholders involved in program implementation. Data were analysed using inductive content analysis. Aboriginal Maternal and Infant Health Service Data Collection (AMDC) records for the period July 2012-June 2015 were investigated to examine how many eligible women attended a service implementing QFNL and how many women took up a QFNL support. Smoking cessation rates were compared in women attending a service offering QFNL with women attending the same service prior to the implementation of QFNL to determine program impact. RESULTS: QFNL was implemented in 70 services located in 13 LHDs across New South Wales. Over 430 staff attended QFNL training, including 101 staff in Aboriginal-identified roles. In the period July 2012-June 2015 27% (n = 1549) of eligible women attended a service implementing QFNL and 21% (n = 320) of these were recorded as taking up a QFNL support. While stakeholders shared stories of success, no statistically significant impact of QFNL on smoking cessation rates was identified (N = 3502; Odds ratio (OR) = 1.28; 95% Confidence Interval (CI) = 0.96-1.70; p-value = 0.0905). QFNL was acceptable to both clients and stakeholders, increased awareness about smoking cessation, and gave staff resources to support clients. CONCLUSION: QFNL was perceived as acceptable by stakeholders and clients and provided care providers with knowledge and tangible support to offer women who presented at antenatal care as smokers, however, no statistically significant impact on rates of smoking cessation were found using the measures available.

From trials to communities: implementation and scale-up of health behaviour interventions.

BACKGROUND: To maximise their potential benefits to communities, effective health behaviour interventions need to be implemented, ideally 'at scale', and are often adapted as part of this. To inform future implementation and scale-up efforts, this study broadly sought to understand (i) how often health behaviour interventions are implemented in communities, (ii) the adaptations that occur; (iii) how frequency it occurred 'at scale'; and (iv) factors associated with 'scale-up'. METHODS: A cross-sectional survey was conducted of corresponding authors of trials (randomised or non-randomised) assessing the effects of preventive health behaviour interventions. Included studies of relevant Cochrane reviews served as a sampling frame. Participants were asked to report on the implementation and scale-up (defined as investment in large scale delivery by a (non)government organisation) of their intervention in the community following trial completion, adaptations made, and any research dissemination strategies employed. Information was extracted from published reports of the trial including assessments of effectiveness and risk of bias. RESULTS: Authors of 104 trials completed the survey. Almost half of the interventions were implemented following trial completion (taking on average 19 months), and 54% of those were adapted prior to doing so. The most common adaptations were adding intervention components, and adapting the intervention to fit within the local service setting. Scale-up occurred in 33% of all interventions. There were no significant associations between research trial characteristics such as intervention effectiveness, risk of bias, setting, involvement of end-user, and incidence of scale-up. However the number of research dissemination strategies was positively associated to the odds of an intervention being scaled-up (OR = 1.50; 95% CI: 1.19, 1.88; p < 0.001). CONCLUSIONS: Adaptation of implemented trials is often undertaken. Most health behaviour interventions are not implemented or scaled-up following trial completion. The use of a greater number of dissemination strategies may increase the likelihood of scaled up.

Disseminating health research to public health policy-makers and practitioners: a survey of source, message content and delivery modality preferences.

BACKGROUND: Understanding the views of policy-makers and practitioners regarding how best to communicate research evidence is important to support research use in their decision-making. AIM: To quantify and describe public health policy-makers and practitioners' views regarding the source, content and form of messages describing public health research findings to inform their decision-making. We also sought to examine differences in preferences between public health policy-makers and practitioners. METHODS: A cross sectional, value-weighting survey of policy-makers and practitioners was conducted. Participants were asked to allocate a proportion of 100 points across different (i) sources of research evidence, (ii) message content and (iii) the form in which evidence is presented. Points were allocated based on their rating of influence, usefulness and preference when making decisions about health policy or practice. RESULTS: A total of 186 survey responses were received from 90 policy-makers and 96 practitioners. Researchers and government department agencies were the most influential source of research evidence based on mean allocation of points, followed by knowledge brokers, professional peers and associations. Mean point allocation for perceived usefulness of message content was highest for simple summary of key findings and implications, and then evidence-based recommendations and data and statistical summaries. Finally, based on mean scores, policy-makers and practitioners preferred to receive research evidence in the form of peer-reviewed publications, reports, evidence briefs and plain language summaries. There were few differences in scores between policy-makers and practitioners across source, message content or form assessments or those with experience in different behavioural areas. CONCLUSIONS: The findings should provide a basis for the future development and optimization of dissemination strategies to this important stakeholder group.

Team characteristics associated with weight loss in a First Nations community intervention: An observational study.

ISSUE ADDRESSED: Group-based weight-loss programs can be effective in addressing high rates of overweight and obesity among Aboriginal and Torres Strait Islander Peoples. The purpose was to determine associations between demographic and baseline weight-related variables and team weight loss in a community-based intervention as no previous studies have analysed this at a team level. METHODS: Binomial models tested associations between team-level age, proportion female and baseline weight and classification as higher weight-loss team (HWT) (>50% persons losing 2.5% of initial weight) vs lower weight-loss team (LWT). Linear regressions compared HWT and LWT on diet and physical activity (PA) outcomes adjusted for age and gender. RESULTS: For each 1 kg increment in mean baseline weight, a team's likelihood of higher weight loss was increased by 4% (APR: 1.04, 95%CI: 1.00, 1.08). HWTs increased vigorous PA by 0.32 sessions more than LWTs (P = .02). Fruit and vegetable intakes were not associated with team weight loss classification. CONCLUSIONS: Only baseline weight and vigorous PA distinguished HWT and LWT. Promoting PA components in team-based weight-loss approaches may be beneficial as these lend themselves to group participation. SO WHAT?: Demographic and baseline weight-related variables are largely not predictive of weight loss success in group programs. Identifying other characteristics shared by HWT may help teams achieve weight loss.

Tools for assessing the scalability of innovations in health: a systematic review.

BACKGROUND: The last decade has seen growing interest in scaling up of innovations to strengthen healthcare systems. However, the lack of appropriate methods for determining their potential for scale-up is an unfortunate global handicap. Thus, we aimed to review tools proposed for assessing the scalability of innovations in health. METHODS: We conducted a systematic review following the COSMIN methodology. We included any empirical research which aimed to investigate the creation, validation or interpretability of a scalability assessment tool in health. We searched Embase, MEDLINE, CINAHL, Web of Science, PsycINFO, Cochrane Library and ERIC from their inception to 20 March 2019. We also searched relevant websites, screened the reference lists of relevant reports and consulted experts in the field. Two reviewers independently selected and extracted eligible reports and assessed the methodological quality of tools. We summarized data using a narrative approach involving thematic syntheses and descriptive statistics. RESULTS: We identified 31 reports describing 21 tools. Types of tools included criteria (47.6%), scales (33.3%) and checklists (19.0%). Most tools were published from 2010 onwards (90.5%), in open-access sources (85.7%) and funded by governmental or nongovernmental organizations (76.2%). All tools were in English; four were translated into French or Spanish (19.0%). Tool creation involved single (23.8%) or multiple (19.0%) types of stakeholders, or stakeholder involvement was not reported (57.1%). No studies reported involving patients or the public, or reported the sex of tool creators. Tools were created for use in high-income countries (28.6%), low- or middle-income countries (19.0%), or both (9.5%), or for transferring innovations from low- or middle-income countries to high-income countries (4.8%). Healthcare levels included public or population health (47.6%), primary healthcare (33.3%) and home care (4.8%). Most tools provided limited information on content validity (85.7%), and none reported on other measurement properties. The methodological quality of tools was deemed inadequate (61.9%) or doubtful (38.1%). CONCLUSIONS: We inventoried tools for assessing the scalability of innovations in health. Existing tools are as yet of limited utility for assessing scalability in health. More work needs to be done to establish key psychometric properties of these tools. Trial registration We registered this review with PROSPERO (identifier: CRD42019107095).

Progress Towards Elimination of Hepatitis C Infection Among People Who Inject Drugs in Australia: The ETHOS Engage Study.

BACKGROUND: Evaluating progress towards hepatitis C virus (HCV) elimination is critical. This study estimated prevalence of current HCV infection and HCV treatment uptake among people who inject drugs (PWID) in Australia. METHODS: The Enhancing Treatment of Hepatitis C in Opioid Substitution Settings Engage is an observational study of PWID attending drug treatment clinics and needle and syringe programs (NSPs). Participants completed a questionnaire including self-reported treatment history and underwent point-of-care HCV RNA testing (Xpert HCV Viral Load Fingerstick; Cepheid). RESULTS: Between May 2018 and September 2019, 1443 participants were enrolled (64% injected drugs in the last month, 74% receiving opioid agonist therapy [OAT]). HCV infection status was uninfected (28%), spontaneous clearance (16%), treatment-induced clearance (32%), and current infection (24%). Current HCV was more likely among people who were homeless (adjusted odds ratio, 1.47; 95% confidence interval, 1.00-2.16), incarcerated in the previous year (2.04; 1.38-3.02), and those injecting drugs daily or more (2.26; 1.43-2.42). Among those with previous chronic or current HCV, 66% (n = 520/788) reported HCV treatment. In adjusted analysis, HCV treatment was lower among females (.68; .48-.95), participants who were homeless (.59; .38-.96), and those injecting daily or more (.51; .31-.89). People aged ≥45 years (1.46; 1.06-2.01) and people receiving OAT (2.62; 1.52-4.51) were more likely to report HCV treatment. CONCLUSIONS: Unrestricted direct-acting antiviral therapy access in Australia has yielded high treatment uptake among PWID attending drug treatment and NSPs, with a marked decline in HCV prevalence. To achieve elimination, PWID with greater marginalization may require additional support and tailored strategies to enhance treatment.

How effective are physical activity interventions when they are scaled-up: a systematic review.

BACKGROUND: The 'scale-up' of effective physical activity interventions is required if they are to yield improvements in population health. The purpose of this study was to systematically review the effectiveness of community-based physical activity interventions that have been scaled-up. We also sought to explore differences in the effect size of these interventions compared with prior evaluations of their efficacy in more controlled contexts, and describe adaptations that were made to interventions as part of the scale-up process. METHODS: We performed a search of empirical research using six electronic databases, hand searched reference lists and contacted field experts. An intervention was considered 'scaled-up' if it had been intentionally delivered on a larger scale (to a greater number of participants, new populations, and/or by means of different delivery systems) than a preceding randomised control trial ('pre-scale') in which a significant intervention effect (p < 0.05) was reported on any measure of physical activity. Effect size differences between pre-scale and scaled up interventions were quantified ([the effect size reported in the scaled-up study / the effect size reported in the pre-scale-up efficacy trial] × 100) to explore any scale-up 'penalties' in intervention effects. RESULTS: We identified 10 eligible studies. Six scaled-up interventions appeared to achieve significant improvement on at least one measure of physical activity. Six studies included measures of physical activity that were common between pre-scale and scaled-up trials enabling the calculation of an effect size difference (and potential scale-up penalty). Differences in effect size ranged from 132 to 25% (median = 58.8%), suggesting that most scaled-up interventions typically achieve less than 60% of their pre-scale effect size. A variety of adaptations were made for scale-up - the most common being mode of delivery. CONCLUSION: The majority of interventions remained effective when delivered at-scale however their effects were markedly lower than reported in pre-scale trials. Adaptations of interventions were common and may have impacted on the effectiveness of interventions delivered at scale. These outcomes provide valuable insight for researchers and public health practitioners interested in the design and scale-up of physical activity interventions, and contribute to the growing evidence base for delivering health promotion interventions at-scale. TRIAL REGISTRATION: PROSPERO CRD42020144842 .

A cost-benefit analysis framework for preventive health interventions to aid decision-making in Australian governments.

BACKGROUND: Australian governments are increasingly mandating the use of cost-benefit analysis (CBA) to inform the efficient allocation of government resources. CBA is likely to be useful when evaluating preventive health interventions that are often cross-sectoral in nature and require Cabinet approval prior to implementation. This study outlines a CBA framework for the evaluation of preventive health interventions that balances the need for consistency with other agency guidelines whilst adhering to guidelines and conventions for health economic evaluations. METHODS: We analysed CBA and other evaluation guidance documents published by Australian federal and New South Wales (NSW) government departments. Data extraction compared the recommendations made by different agencies and the impact on the analysis of preventive health interventions. The framework specifies a reference case and sensitivity analyses based on the following considerations: (1) applied economic evaluation theory; (2) consistency between CBA across different government departments; (3) the ease of moving from a CBA to a more conventional cost-effectiveness/cost-utility analysis framework often used for health interventions; (4) the practicalities of application; and (5) the needs of end users being both Cabinet decision-makers and health policy-makers. RESULTS: Nine documents provided CBA or relevant economic evaluation guidance. There were differences in terminology and areas of agreement and disagreement between the guidelines. Disagreement between guidelines involved (1) the community included in the societal perspective; (2) the number of options that should be appraised in ex ante analyses; (3) the appropriate time horizon for interventions with longer economic lives; (4) the theoretical basis and value of the discount rate; (5) parameter values for variables such as the value of a statistical life; and (6) the summary measure for decision-making. CONCLUSIONS: This paper addresses some of the methodological challenges that have hindered the use of CBA in prevention by outlining a framework that is consistent with treasury department guidelines whilst considering the unique features of prevention policies. The effective use and implementation of a preventive health CBA framework is likely to require considerable investment of time and resources from state and federal government departments of health and treasury but has the potential to improve decision-making related to preventive health policies and programmes.

The how and why of producing policy relevant research: perspectives of Australian childhood obesity prevention researchers and policy makers.

BACKGROUND: Understanding why research is conducted may help address the under-utilisation of research. This study examined the reasons for childhood obesity prevention knowledge production in New South Wales (NSW), Australia, and the factors influencing research choices from the perspective of the researchers and health policy agencies contributing to the research. METHODS: A literature search of SCOPUS and ISI Web of Knowledge (affiliation and key word searches) was conducted to compile a database of NSW childhood obesity research outputs, published between 2000 and 2015 (n = 543). Descriptive statistics were used to quantify outputs by research type, differentiating measurement, descriptive, and intervention research, systematic reviews and other publications. Interviews were conducted with a sample of researchers drawn from the database (n = 13) and decision makers from health policy agencies who funded and contributed to childhood obesity research in NSW (n = 15). Researcher interviews examined views about societal impacts, why and under what circumstances the research was conducted. Decision-maker interviews examined policy agency research investment and how research was used in decision making. Content analysis and a thematic approach was used to analyse the interview transcripts. RESULTS: The research in this case was conducted for mix of reasons including those traditionally associated with academic inquiry, as well as intentions to influence policy and practice. Differences in funding mechanisms, administrative and employment arrangements, and 'who' initiated the research, created differing incentives and perspectives for knowledge production. Factors associated with the characteristics and experience of the individuals involved also influenced goals, as did the type of research conducted. Policy agencies played a role in directing research to address policy needs. CONCLUSIONS: The findings of this study confirm that researchers are strongly influenced by their working environment. Funding schemes and other incentives to support policy relevant knowledge production are important. Contextual factors such as policy priorities, policy-driven research funding and the embedded nature or strong connections between some researchers and the policy agencies involved, are likely to have influenced the extent to which policy goals were reported in this study.

The Weight of Words: Co-Analysis of Thick Ethnographic Description and "Friction" as Methodological Strategies in a Health Policy Research Partnership.

Co-production partnerships between policymakers, practitioners, and researchers are designed to facilitate production of relevant and readily usable research in health policy and practice contexts. We describe methodological strategies for in-depth collaborative analysis based on a co-produced ethnography of health promotion practice, involving ethnographic researchers and government-based research partners. We draw on a co-production dialogue to reflect critically on the role and value of co-analyzing research findings using thick ethnographic descriptions. The ambiguity of ethnographic imagery allowed flexibility in interpretation of findings and also generated friction. Specific ethnographic images became focal points for productive friction that crystallized ethical and analytical imperatives underpinning the diverse expertise in the team. To make the most of co-analysis of thick ethnographic descriptions, we assert that friction points must be reflexively considered as key learning opportunities for (a) higher order analysis informed by diverse analytical perspectives and (b) more cohesive and useful interpretations of research findings.

Identification and evaluation of risk of generalizability biases in pilot versus efficacy/effectiveness trials: a systematic review and meta-analysis.

BACKGROUND: Preliminary evaluations of behavioral interventions, referred to as pilot studies, predate the conduct of many large-scale efficacy/effectiveness trial. The ability of a pilot study to inform an efficacy/effectiveness trial relies on careful considerations in the design, delivery, and interpretation of the pilot results to avoid exaggerated early discoveries that may lead to subsequent failed efficacy/effectiveness trials. "Risk of generalizability biases (RGB)" in pilot studies may reduce the probability of replicating results in a larger efficacy/effectiveness trial. We aimed to generate an operational list of potential RGBs and to evaluate their impact in pairs of published pilot studies and larger, more well-powered trial on the topic of childhood obesity. METHODS: We conducted a systematic literature review to identify published pilot studies that had a published larger-scale trial of the same or similar intervention. Searches were updated and completed through December 31st, 2018. Eligible studies were behavioral interventions involving youth (≤18 yrs) on a topic related to childhood obesity (e.g., prevention/treatment, weight reduction, physical activity, diet, sleep, screen time/sedentary behavior). Extracted information included study characteristics and all outcomes. A list of 9 RGBs were defined and coded: intervention intensity bias, implementation support bias, delivery agent bias, target audience bias, duration bias, setting bias, measurement bias, directional conclusion bias, and outcome bias. Three reviewers independently coded for the presence of RGBs. Multi-level random effects meta-analyses were performed to investigate the association of the biases to study outcomes. RESULTS: A total of 39 pilot and larger trial pairs were identified. The frequency of the biases varied: delivery agent bias (19/39 pairs), duration bias (15/39), implementation support bias (13/39), outcome bias (6/39), measurement bias (4/39), directional conclusion bias (3/39), target audience bias (3/39), intervention intensity bias (1/39), and setting bias (0/39). In meta-analyses, delivery agent, implementation support, duration, and measurement bias were associated with an attenuation of the effect size of - 0.325 (95CI - 0.556 to - 0.094), - 0.346 (- 0.640 to - 0.052), - 0.342 (- 0.498 to - 0.187), and - 0.360 (- 0.631 to - 0.089), respectively. CONCLUSIONS: Pre-emptive avoidance of RGBs during the initial testing of an intervention may diminish the voltage drop between pilot and larger efficacy/effectiveness trials and enhance the odds of successful translation.

Weight change among repeat participants of an Aboriginal community-based weight loss program.

BACKGROUND: Community-based weight loss programs may have potential to address overweight and obesity at the population level. However, participation patterns and individual outcomes from these programs are understudied. This study examined repeat participation patterns and participant weight change between contests over seven years of an Aboriginal Australian team-based program in order to identify (1) predictors of repeat participation and (2) associations with weight change between contests. METHODS: Data for the 12 contests from 2012 to 2018 were merged, with probabilistic record matching. A total of 7510 enrolments were registered for the 12 contests, representing 4438 unique people. Contest lengths varied from 10 to 16 weeks in duration. Non-repeat participants were those who only competed once in the program by the end of 2018, and repeaters were those who competed in at least two contests. Associations between repeat participation and participant baseline (i.e., first participation occasion) characteristics, change in diet and physical activity and percent change in weight during the first participation occasion were examined using crossed random effects (for person and team) regression adjusted for exposure to the program. Weight percentage change between contests was calculated for consecutive participation occasions occurring at least three months apart, converted to percent change per month. Weight change was regressed on number of repeat participation occasions adjusted for age, gender, baseline weight at first participation occasion, and weight percent change in the immediately preceding contest. RESULTS: One-third of the 4433 participants participated more than once, with women more likely than men to repeat. A 1% reduction in weight during a competition was associated with an increase in weight of 0.05% per month between competition end and subsequent participation. Regain was smaller the heavier participants were at their first participation. CONCLUSIONS: While individuals benefit from weight loss through program participation, strengthening strategies for weight loss maintenance within or following the program could improve long-term weight outcomes and reduce weight cycling.

Can an electronic monitoring system capture implementation of health promotion programs? A focussed ethnographic exploration of the story behind program monitoring data.

BACKGROUND: There is a pressing need for policy makers to demonstrate progress made on investments in prevention, but few examples of monitoring systems capable of tracking population-level prevention policies and programs and their implementation. In New South Wales, Australia, the scale up of childhood obesity prevention programs to over 6000 childcare centres and primary schools is monitored via an electronic monitoring system, "PHIMS". METHODS: Via a focussed ethnography with all 14 health promotion implementation teams in the state, we set out to explore what aspects of program implementation are captured via PHIMS, what aspects are not, and the implications for future IT implementation monitoring systems as a result. RESULTS: Practitioners perform a range of activities in the context of delivering obesity prevention programs, but only specific activities are captured via PHIMS. PHIMS thereby defines and standardises certain activities, while non-captured activities can be considered as "extra" work by practitioners. The achievement of implementation targets is influenced by multi-level contextual factors, with only some of the factors accounted for in PHIMS. This evidences incongruencies between work done, recorded and, therefore, recognised. CONCLUSIONS: While monitoring systems cannot and should not capture every aspect of implementation, better accounting for aspects of context and "extra" work involved in program implementation could help illuminate why implementation succeeds or fails. Failure to do so may result in policy makers drawing false conclusions about what is required to achieve implementation targets. Practitioners, as experts of context, are well placed to assist policy makers to develop accurate and meaningful implementation targets and approaches to monitoring.

Adaptation of public health initiatives: expert views on current guidance and opportunities to advance their application and benefit.

While there is some guidance to support the adaptation of evidence-based public health interventions, little is known about adaptation in practice and how to best support public health practitioners in its operationalization. This qualitative study was undertaken with researchers, methodologists, policy makers and practitioners representing public health expert organizations and universities internationally to explore their views on available adaptation frameworks, elicit potential improvements to such guidance, and identify opportunities to improve implementation of public health initiatives. Participants attended a face to face workshop in Newcastle, Australia in October 2018 where World Café and focus group discussions using Appreciative Inquiry were undertaken. A number of limitations with current guidance were reported, including a lack of detail on 'how' to adapt, limited information on adaptation of implementation strategies and a number of structural issues related to the wording and ordering of elements within frameworks. A number of opportunities to advance the field was identified. Finally, a list of overarching principles that could be applied together with existing frameworks was generated and suggested to provide a practical way of supporting adaptation decisions in practice.

Scaling up population health interventions from decision to sustainability - a window of opportunity? A qualitative view from policy-makers.

BACKGROUND: While known efficacious preventive health interventions exist, the current capacity to scale up these interventions is limited. In recent years, much attention has focussed on developing frameworks and methods for scale-up yet, in practice, the pathway for scale-up is seldom linear and may be highly dependent on contextual circumstances. Few studies have examined the process of scaling up from decision to implementation nor examined the sustainability of scaled-up interventions. This study explores decision-makers' perceptions from real-world scaled-up case studies to examine how scale-up decisions were made and describe enablers of successful scale-up and sustainability. METHODS: This qualitative study included 29 interviews conducted with purposively sampled key Australian policy-makers, practitioners and researchers experienced in scale-up. Semi-structured interview questions obtained information regarding case studies of scaled-up interventions. The Framework Analysis method was used as the primary method of analysis of the interview data to inductively generate common and divergent themes within qualitative data across cases. RESULTS: A total of 31 case studies of public health interventions were described by interview respondents based on their experiences. According to the interviewees' perceptions, decisions to scale up commonly occurred either opportunistically, when funding became available, or when a deliberate decision was made and funding allocated. The latter scenario was more common when the intervention aligned with specific political or strategic goals. Decisions to scale up were driven by a variety of key actors such as politicians, senior policy-makers and practitioners in the health system. Drivers of a successful scale-up process included good governance, clear leadership, and adequate resourcing and expertise. Establishing accountability structures and appropriate engagement mechanisms to encourage the uptake of interventions were also key enablers. Sustainability was influenced by evidence of impact as well as good acceptability among the general or target population. CONCLUSIONS: Much like Kingdon's Multiple Streams Theory of 'policy windows', there is a conceptually similar 'window for scale-up', driven by a complex interplay of factors such as political need, strategic context, funding and key actors. Researchers and policy-makers need to consider scalability from the outset and prepare for when the window for scale-up opens. Decision-makers need to provide longer term funding for scale-up to facilitate longer term sustainability and build on the resources already invested for the scale-up process.

Intervention Scalability Assessment Tool: A decision support tool for health policy makers and implementers.

BACKGROUND: Promising health interventions tested in pilot studies will only achieve population-wide impact if they are implemented at scale across communities and health systems. Scaling up effective health interventions is vital as not doing so denies the community the most effective services and programmes. However, there remains a paucity of practical tools to assess the suitability of health interventions for scale-up. The Intervention Scalability Assessment Tool (ISAT) was developed to support policy-makers and practitioners to make systematic assessments of the suitability of health interventions for scale-up. METHODS: The ISAT was developed over three stages; the first stage involved a literature review to identify similar tools and frameworks that could be used to guide scalability assessments, and expert input to develop draft ISAT content. In the second stage, the draft ISAT tool was tested with end users. The third stage involved revising and re-testing the ISAT with end users to further refine the language and structure of the final ISAT. RESULTS: A variety of information and sources of evidence should be used to complete the ISAT. The ISAT consists of three parts. Part A: 'setting the scene' requires consideration of the context in which the intervention is being considered for scale-up and consists of five domains, as follows: (1) the problem; (2) the intervention; (3) strategic/political context; (4) evidence of effectiveness; and (5) intervention costs and benefits. Part B asks users to assess the potential implementation and scale-up requirements within five domains, namely (1) fidelity and adaptation; (2) reach and acceptability; (3) delivery setting and workforce; (4) implementation infrastructure; and (5) sustainability. Part C generates a graphical representation of the strengths and weaknesses of the readiness of the proposed intervention for scale-up. Users are also prompted for a recommendation as to whether the intervention (1) is recommended for scale-up, (2) is promising but needs further information before scaling up, or (3) does not yet merit scale-up. CONCLUSION: The ISAT fills an important gap in applied scalability assessment and can become a critical decision support tool for policy-makers and practitioners when selecting health interventions for scale-up. Although the ISAT is designed to be a health policy and practitioner tool, it can also be used by researchers in the design of research to fill important evidence gaps.