Food Systems Interventions for Nutrition: Lessons from 6 Program Evaluations in Africa and South Asia.

Although there is growing global momentum behind food systems strategies to improve planetary and human health-including nutrition-there is limited evidence of what types of food systems interventions work. Evaluating these types of interventions is challenging due to their complex and dynamic nature and lack of fit with standard evaluation methods. In this article, we draw on a portfolio of 6 evaluations of food systems interventions in Africa and South Asia that were intended to improve nutrition. We identify key methodological challenges and formulate recommendations to improve the quality of such studies. We highlight 5 challenges: a lack of evidence base to justify the intervention, the dynamic and multifaceted nature of the interventions, addressing attribution, collecting or accessing accurate and timely data, and defining and measuring appropriate outcomes. In addition to more specific guidance, we identify 6 cross-cutting recommendations, including a need to use multiple and diverse methods and flexible designs. We also note that these evaluation challenges present opportunities to develop new methods and highlight several specific needs in this space.

Overcoming Common Anxieties in Knowledge Translation: Advice for Scholarly Issue Advocates.

Policy Points Faced with urgent threats to human health and well-being such as climate change, calls among the academic community are getting louder to contribute more effectively to the implementation of the evidence generated by our research into public policy. As interest in knowledge translation (KT) surges, so have a number of anxieties about the field's shortcomings. Our paper is motivated by a call in the literature to render useful advice for those beginning in KT on how to advance impact at a policy level. By integrating knowledge from fields such as political science, moral psychology, and marketing, we suggest that thinking and acting like marketers, lobbyists, movements, and political scientists would help us advance on the quest to bridge the chasm between evidence and policy.

Using evidence-based co-design to develop a hybrid delivered exercise intervention that aims to increase confidence to exercise in people with haemophilia.

INTRODUCTION: Due to advances in treatments, people with haemophilia (PWH) are living longer. They are not as active as the general population due to joint damage and lack confidence to be active due to concerns about further bleeds and pain. There is a need to facilitate healthy aging through promotion of physical activity (PA) and exercise. Changing patient beliefs and increasing physical literacy and confidence to move are thought to be key to helping PWH become more active. AIM: This paper describes the development of an exercise and behaviour change intervention to improve confidence to exercise in PWH. METHODS: The 4-stage Medical Research Council framework for complex intervention development was used. RESULTS: Stakeholders included 17 PWH and 7 physiotherapists working in haemophilia. Seven online focus group meetings were held. The final intervention is a hybrid 12-week physiotherapist led progressive exercise programme. Classes are 45 min including Pilates, High intensity interval training and balance elements, together with discussion sessions focusing on PA recommendations, the types and benefits of different exercise styles and the effects of PA, together with the effects of aging for PWH. The COM-B model of behaviour change was used to develop the intervention. CONCLUSION: Co-design helps to produce an intervention that understands the stakeholders needs. Through this process the intervention developed to incorporate not only increasing PA but also confidence to exercise. The use of behaviour change theory identified the behaviour techniques included in the intervention and aims to increase physical literacy in this population.

Implementation of peer support for people with severe mental health conditions in high-, middle- and low-income-countries: a theory of change approach.

BACKGROUND: Stakeholder engagement is essential to the design, implementation and evaluation of complex mental health interventions like peer support. Theory of Change (ToC) is commonly used in global health research to help structure and promote stakeholder engagement throughout the project cycle. Stakeholder insights are especially important in the context of a multi-site trial, in which an intervention may need to be adapted for implementation across very different settings while maintaining fidelity to a core model. This paper describes the development of a ToC for a peer support intervention to be delivered to people with severe mental health conditions in five countries as part of the UPSIDES trial. METHODS: One hundred thirty-four stakeholders from diverse backgrounds participated in a total of 17 workshops carried out at six UPSIDES implementing sites across high-, middle- and low-income settings (one site each in India, Israel, Uganda and Tanzania; two sites in Germany). The initial ToC maps created by stakeholders at each site were integrated into a cross-site ToC map, which was then revised to incorporate additional insights from the academic literature and updated iteratively through multiple rounds of feedback provided by the implementers. RESULTS: The final ToC map divides the implementation of the UPSIDES peer support intervention into three main stages: preparation, implementation, and sustainability. The map also identifies three levels of actors involved in peer support: individuals (service users and peer support workers), organisations (and their staff members), and the public. In the UPSIDES trial, the ToC map proved especially helpful in characterising and distinguishing between (a) common features of peer support, (b) shared approaches to implementation and (c) informing adaptations to peer support or implementation to account for contextual differences. CONCLUSIONS: UPSIDES is the first project to develop a multi-national ToC for a mental health peer support intervention. Stakeholder engagement in the ToC process helped to improve the cultural and contextual appropriateness of a complex intervention and ensure equivalence across sites for the purposes of a multi-site trial. It may serve as a blueprint for implementing similar interventions with a focus on recovery and social inclusion among people with mental ill-health across diverse settings. TRIAL REGISTRATION: ISRCTN26008944 (Registration Date: 30/10/2019).

The global challenge of reducing mercury contamination from artisanal and small-scale gold mining (ASGM): evaluating solutions using generic theories of change.

Mercury contamination from artisanal and small-scale gold mining (ASGM) currently accounts for 37% of the global total, often affecting tropical regions where regulations, if they exist, are often poorly enforced. Ingestion by people and other animals damages the nervous, reproductive, and cognitive systems. Despite the efforts of many organizations and governments to curb mercury releases from ASGM, it is increasing globally. There are many possible interventions, all with significant complexity and cost. Therefore, we recommend taking an established systematic approach to articulate the current situation and construct theories of change (ToC) for different possible interventions for any government or organization trying to solve this problem. Here we present a high-level situation analysis and generic ToC to support a more coordinated approach that explicitly builds upon previous experience to identify organization- and situation-appropriate engagement on this issue. We then illustrate the use of these generic models to construct a specific ToC with a policy-focused entry point. This includes interventions through (1) engagement with the global Minamata Convention on Mercury; (2) support for existing national laws and policies connected to ASGM and mercury contamination; and (3) engagement of indigenous people and local communities with governments to meet the governments' legal obligations. By methodically articulating assumptions about interventions, connections among actions, and desired outcomes, it is possible to create a more effective approach that will encourage more coordination and cooperation among governments and other practitioners to maximize their investments and support broad environmental and socio-political outcomes necessary to address this pernicious problem.

Adapting brief problem-solving therapy for pregnant women experiencing depressive symptoms and intimate partner violence in rural Ethiopia.

OBJECTIVE: To adapt an evidence-based psychological intervention for pregnant women experiencing depressive symptoms and intimate partner violence (IPV) in rural Ethiopia. METHOD: We conducted a desk review of contextual factors in Sodo, Ethiopia, followed by qualitative interviews with 16 pregnant women and 12 antenatal care (ANC) providers. We engaged stakeholders through participatory theory of change (ToC) workshops, to select the intervention and articulate a programme theory. We used "ADAPT" guidance to adapt the intervention to the context, before mapping potential harms in a "dark logic model". RESULTS: Brief problem-solving therapy developed for South Africa was the most contextually relevant model. We adapted the delivery format (participants prioritised confidentiality and brevity) and training and supervision (addressing IPV). Consensus long-term outcomes in our ToC were ANC providers skilled in detecting and responding to emotional difficulties and IPV, women receiving appropriate support, and emotional difficulties improving. Our dark logic model highlighted the risk of more severe IPV and mental health symptoms not being referred appropriately. CONCLUSION: Although intervention adaptation is recommended, the process is rarely reported in depth. We comprehensively describe how contextual considerations, stakeholder engagement, programme theory, and adaptation can tailor psychological interventions for the target population in a low-income, rural setting.

Theory of change and strategic priorities of the world federation of haemophilia.

The World Federation of Haemophilia (WFH) is a global network of national member organizations (NMOs) that advocate, collectively and individually, to improve lives of people with inherited bleeding disorders. The WFH vision of "Treatment for All" speaks to a future in which all people with an inherited bleeding disorder will have access to care, regardless of their gender or where they live. Over the last several years, initiatives including the WFH Humanitarian Aid program, the World Bleeding Disorders Registry, and Guidelines for the Management of Haemophilia and von Willebrand disease have significantly changed how the WFH and its partners work to improve and sustain care for people with bleeding disorders. Following an extensive consultation that included over 200 stakeholders from 70 countries, a Theory of Change was developed, and strategic priorities identified, to clearly define the WFH's intended impact and point of accountability to its stakeholders, and to determine how and through who those goals will be achieved. Both should help the WFH better support its NMOs and healthcare providers around the world in their efforts to improve access to diagnosis and care, as new therapies revolutionize the treatment landscape and the fallout of the global pandemic continues to challenge the ways in which we work and connect. Global collaboration of all stakeholders, based on their resources, objectives and skills, will be required to achieve these goals and to ensure more people have reliable access to safe treatment and care, regardless of their bleeding disorder, gender, or where they live.

A guide to qualitative attribution methods for evaluation in conservation.

Knowledge of what conservation interventions improve biodiversity outcomes, and in which circumstances, is imperative. Experimental and quasi-experimental methods are increasingly used to establish causal inference and build the evidence base on the effectiveness of interventions, but their ability to provide insight into how and under what conditions an intervention should be implemented to improve biodiversity outcomes faces limitations. A suite of attribution methods that leverage qualitative methods for causal inference is available but underutilized in conversation impact evaluation. This article provides a guide to 5 such qualitative attribution methods: contribution analysis, process tracing, realist evaluation, qualitative comparative analysis, and most significant change. It defines and introduces each method and then illustrates how they could be applied through a case study of community conservancies in Namibia. This guide provides examples of how qualitative attribution methods can advance knowledge of what works, in which contexts, and why in biodiversity conservation.

Guía para los métodos de atribución cualitativa para la evaluación en conservación Resumen El conocimiento sobre cuáles intervenciones de conservación son las que aumentan los resultados en biodiversidad y en cuáles circunstancias sucede esto es imperativo. Los métodos experimentales y semiexperimentales se usan cada vez más para establecer la inferencia casual y construir la base de evidencias sobre la efectividad de las intervenciones, pero su capacidad para proporcionar conocimiento sobre cómo y bajo cuáles condiciones se debería implementar una intervención para aumentar los resultados en biodiversidad enfrenta limitaciones. Un conjunto de métodos de atribución que impulse los métodos cualitativos para la inferencia casual se encuentra disponible pero poco utilizado en la evaluación de los impactos de conservación. Este artículo proporciona una guía para cinco de los métodos de atribución cualitativa: análisis de contribución, rastreo de procesos, evaluación realista, análisis cualitativo comparativo y el cambio más significativo. La guía define e introduce cada método y después ilustra cómo podrían aplicarse mediante un estudio de caso de conservación en Namibia. Esta guía proporciona ejemplos de cómo los métodos de atribución cualitativa pueden incrementar el conocimiento de qué funciona, en cuáles contextos y por qué en la conservación d la biodiversidad.

Enhancing the quality of psychological interventions delivered by telephone in mental health services: increasing the likelihood of successful implementation using a theory of change.

BACKGROUND: The implementation of new and complex interventions in mental health settings can be challenging. This paper explores the use of a Theory of Change (ToC) for intervention design and evaluation to increase the likelihood of complex interventions being effective, sustainable, and scalable. Our intervention was developed to enhance the quality of psychological interventions delivered by telephone in primary care mental health services. METHODS: A ToC represents how our designed quality improvement intervention targeting changes at service, practitioner, and patient levels was expected to improve engagement in, and the quality of, telephone-delivered psychological therapies. The intervention was evaluated following implementation in a feasibility study within three NHS Talking Therapies services through a qualitative research design incorporating semi-structured interviews and a focus group with key stakeholders (patients, practitioners, and service leads) (N = 15). Data were analysed using the Consolidated Framework for Implementation Research (CFIR) and the ToC was examined and modified accordingly following the findings. RESULTS: CFIR analysis highlighted a set of challenges encountered during the implementation of our service quality improvement telephone intervention that appeared to have weakened the contribution to the change mechanisms set out by the initial ToC. Findings informed changes to the intervention and refinement of the ToC and are expected to increase the likelihood of successful future implementation in a randomised controlled trial. CONCLUSIONS: Four key recommendations that could help to optimise implementation of a complex intervention involving different key stakeholder groups in any setting were identified. These include: 1-developing a good understanding of the intervention and its value among those receiving the intervention; 2-maximising engagement from key stakeholders; 3-ensuring clear planning and communication of implementation goals; and 4-encouraging the use of strategies to monitor implementation progress.

Outcome assessment of a complex mental health intervention in the workplace. Results from the MENTUPP pilot study.

OBJECTIVE: Multicomponent interventions are recommendable to achieve the greatest mental health benefits, but are difficult to evaluate due to their complexity. Defining long-term outcomes, arising from a Theory of Change (ToC) and testing them in a pilot phase, is a useful approach to plan a comprehensive and meaningful evaluation later on. This article reports on the pilot results of an outcome evaluation of a complex mental health intervention and examines whether appropriate evaluation measures and indicators have been selected ahead of a clustered randomised control trial (cRCT). METHODS: The MENTUPP pilot is an evidence-based intervention for Small and Medium Enterprises (SMEs) active in three work sectors and nine countries. Based on our ToC, we selected the MENTUPP long-term outcomes, which are reported in this article, are measured with seven validated scales assessing mental wellbeing, burnout, depression, anxiety, stigma towards depression and anxiety, absenteeism and presenteeism. The pilot MENTUPP intervention assessment took place at baseline and at 6 months follow-up. RESULTS: In total, 25 SMEs were recruited in the MENTUPP pilot and 346 participants completed the validated scales at baseline and 96 at follow-up. Three long-term outcomes significantly improved at follow-up (p < 0.05): mental wellbeing, symptoms of anxiety, and personal stigmatising attitudes towards depression and anxiety. CONCLUSIONS: The results of this outcome evaluation suggest that MENTUPP has the potential to strengthen employees' wellbeing and decrease anxiety symptoms and stigmatising attitudes. Additionally, this study demonstrates the utility of conducting pilot workplace interventions to assess whether appropriate measures and indicators have been selected. Based on the results, the intervention and the evaluation strategy have been optimised.

Developing a framework for evaluation: a Theory of Change for complex workplace mental health interventions.

BACKGROUND: There is a gap between the necessity of effective mental health interventions in the workplace and the availability of evidence-based information on how to evaluate them. The available evidence outlines that mental health interventions should follow integrated approaches combining multiple components related to different levels of change. However, there is a lack of robust studies on how to evaluate multicomponent workplace interventions which target a variety of outcomes at different levels taking into account the influence of different implementation contexts. METHOD: We use the MENTUPP project as a research context to develop a theory-driven approach to facilitate the evaluation of complex mental health interventions in occupational settings and to provide a comprehensive rationale of how these types of interventions are expected to achieve change. We used a participatory approach to develop a ToC involving a large number of the project team representing multiple academic backgrounds exploiting in tandem the knowledge from six systematic reviews and results from a survey among practitioners and academic experts in the field of mental health in SMEs. RESULTS: The ToC revealed four long-term outcomes that we assume MENTUPP can achieve in the workplace: 1) improved mental wellbeing and reduced burnout, 2) reduced mental illness, 3) reduced mental illness-related stigma, and 4) reduced productivity losses. They are assumed to be reached through six proximate and four intermediate outcomes according to a specific chronological order. The intervention consists of 23 components that were chosen based on specific rationales to achieve change on four levels (employee, team, leader, and organization). CONCLUSIONS: The ToC map provides a theory of how MENTUPP is expected to achieve its anticipated long-term outcomes through intermediate and proximate outcomes assessing alongside contextual factors which will facilitate the testing of hypotheses. Moreover, it allows for a structured approach to informing the future selection of outcomes and related evaluation measures in either subsequent iterations of complex interventions or other similarly structured programs. Hence, the resulting ToC can be employed by future research as an example for the development of a theoretical framework to evaluate complex mental health interventions in the workplace.

Improving paramedic responses for patients dying at home: a theory of change-based approach.

BACKGROUND: Paramedics are increasingly being called to attend patients dying from advanced incurable conditions. However, confidence to deal with such calls varies, with many feeling relatively unskilled in this aspect of their role. A number of interventions have been piloted to improve their skills in end-of-life care (EoLC) but without a fully specified theoretical model. Theory of Change models can provide theoretical and testable links from intervention activities to proposed long-term outcomes and indicate the areas for assessment of effectiveness. This study aimed to develop an intervention for improving paramedic EoLC for patients in the community. METHODS: A Theory of Change approach was used as the overarching theoretical framework for developing an intervention to improve paramedic end-of-life skills. Nine stakeholders - including specialist community paramedics, ambulance call handlers and palliative care specialists - were recruited to five consecutive online workshops, ranging between 60 and 90 min. Each workshop had 2-3 facilitators. Over multiple workshops, stakeholders decided on the desired impact, short- and long-term outcomes, and possible interventions. During and between these workshops a Theory of Change model was created, with the components shared with stakeholders. RESULTS: The stakeholders agreed the desired impact was to provide consistent, holistic, patient-centred, and effective EoLC. Four potential long-term outcomes were suggested: (1) increased use of anticipatory and regular end-of-life medications; (2) reduced end-of-life clinical and medication errors; (3) reduced unnecessary hospitalisations; (4) increased concordance between patient preferred and actual place of death. Key interventions focused on providing immediate information on what to do in such situations including: appraising the situation, developing an algorithm for a treatment plan (including whether or not to convey to hospital) and how to identify ongoing support in the community. CONCLUSIONS: A Theory of Change approach was effective at identifying impact, outcomes, and the important features of an end-of-life intervention for paramedics. This study identified the need for paramedics to have immediate access to information and resources to support EoLC, which the workshop stakeholders are now seeking to develop as an intervention.

Co-design of dementia prevention program for Aboriginal Australians (DAMPAA).

INTRODUCTION: Dementia is highly prevalent in older Aboriginal Australians, with several modifiable risk factors. Currently, there is limited evidence on how to prevent cognitive decline in Aboriginal Australians. METHODS: Based on our Theory of Change (ToC) framework, we co-developed the Dementia risk management and prevention program for Aboriginal Australians (DAMPAA) aged over 45 years in partnership with Aboriginal community-controlled organizations (ACCOs) and Elders. Qualitative data were collected through ACCO staff workshops, Elders yarning, and governance groups to inform the protocol. Additionally, we conducted a small pilot study. RESULTS: Expected DAMPAA ToC outcomes are: (1) improved daily function, (2) better cardiovascular risk management, (3) falls reduction, (4) improved quality of life, and (5) reduced cognitive decline. Attendance enablers are social interaction, environment, exercise type/level, and logistics. DISCUSSION: Findings suggest that ToC is an effective collaborative approach for co-designing Aboriginal health programs.

A Critical Evaluation of International Agreements Towards a Revised Categorization for Transfrontier Conservation Areas (TFCAs).

Transfrontier Conservation Areas (TFCAs) are widely promoted as an international instrument to achieve certain conservation, cooperation and developmental goals, especially within the Southern African Development Community (SADC). In the SADC context, the status of TFCAs is categorized based on the extent to which international agreements have been signed. These agreements take different forms such as treaties, memorandums of understanding (MoUs), protocols and bilateral agreements. However, the efficacy of agreement-based approaches towards the categorization of TFCAs has been questioned because it does not acknowledge the implementation complexities of TFCAs and lacks a sound conceptual basis. This research evaluates the international TFCA agreements in SADC with a view to recommending a revised categorization. This is achieved by applying Theory of Change (ToC) to a sample of ten signed TFCAs agreements. The results show a lack of enforcement mechanisms, weak provision for implementation and poorly defined objectives. These weaknesses of agreement-based approaches can best be addressed by expanding the categorization of TFCAs to also include the extent of legislative and operational alignment. The revised categorization supports a more complete understanding of TFCA implementation.

A Researcher-Practitioner Collaboration to Enable Replication of an Evidence-Based Financial Education and Coaching Program That Improves Health in Low-Income Single Mothers.

Financial strain is a social determinant of health (SDOH). Although public financial education helps individuals improve financial well-being, specifics are lacking on how and why effective programs work, potentially limiting their successful replication in other practice settings. In this study, researchers and practitioners cocreated the core components and theory of change of a novel financial education and coaching program, which a randomized controlled trial found was effective in significantly improving participants' financial and health-related behaviors. A Cocreating Knowledge Translation Framework within a case study design was used at a university-affiliated nonprofit in Omaha, Nebraska, from August to December 2020. Twelve practitioner and alumni participants were purposefully sampled. An administrative records review, semi-structured interviews (n =3), survey (n = 10), and facilitated backward mapping session (n = 5) were conducted. Transcripts were coded to identify themes. Thirty-one core components were identified within program principles, design, tools, activities, and expectations of participants and coaches. Ten theory of change outcomes described participants' pathway to change. Interventions occurred at individual, relationship, and community levels from initial engagement, through behavioral changes, to improved health-related quality of life. Activities and indicators were mapped to each outcome. The program's intersecting and reinforcing design was key to enabling participants' outcomes. Its theory of change described how and why the model improved financial and health behaviors. Findings suggest that other SDOH-focused organizations may benefit from researcher-practitioner collaboration to investigate their interventions' core components and theories of change. This may enable replication, promoting downstream health benefits in new community settings.

Multi-sectoral action to promote psychological wellbeing: Theorising the role of place-based policy.

ISSUES ADDRESSED: Several national governments are showing interest in policies to promote wellbeing. One common strategy is to devise systems to measure indictors of wellbeing, presuming that governments act on what they measure. This article will argue instead that formation of multisectoral policies to promote psychological wellbeing requires a different kind of theoretical and evidentiary basis. METHODS: The article integrates ideas from literature on wellbeing, health in all policies, political science, mental health promotion, and social determinants of health to make a case for place-based policy as the central feature of multi-sectoral policy for psychological wellbeing. RESULTS AND DISCUSSION: I argue that the required theoretical foundation for policy action on psychological wellbeing lies with understanding certain basic functions of human social psychology including the role of stress arousal. I then draw on policy theory to propose three steps to translate this theoretical understanding of psychological wellbeing into practicable, multi-sectoral policies. Step one is concerned with adopting a thoroughly revised conception of psychological wellbeing as a policy problem. Step two involves uptake of a theory of change in policy, grounded on recognition of essential social conditions required to promote psychological wellbeing. Proceeding from these, I will argue that a necessary (but not sufficient) third step is to implement place-based strategies involving government-community partnerships, to generate essential conditions for psychological wellbeing on a universal basis. Finally, I examine implications of the proposed approach for current theory and practice in mental health promotion policy. CONCLUSIONS: Place-based policy is foundational for effective multi-sectoral policy to promote psychological wellbeing. SO WHAT?: Governments aiming to promote psychological wellbeing should position place-based policy at the centre of their strategies.

Typifying conservation practitioners' views on the role of education.

Education is an established tool to enhance human-environment relationships, despite the lack of empirical evidence to support its use. We used theories of change to unpack assumptions about the role of education in conservation. We interviewed practitioners from 15 conservation organizations in Madagascar to typify implicit pathways of change and assess whether emerging pathways echo theoretical advances. Five pathways were drivers of change: increasing knowledge, changing emotional connection and changing traditional cultural practices, fostering leaders, diversifying outcomes, and influencing community and society. These pathways reflect existing sociopsychological theories on learning and behavioral change. Most interviewees' organizations had a predominant pathway that was often combined with elements from other pathways. Most pathways lacked culturally grounded approaches. Our research reveals assumptions about the role of education in conservation and indicates that organizations had different ideas of how change happens. The diversity of practices reflects the complexity of factors that influence behavior. Whether this diversity is driven by local sociocultural context, interaction with other conservation approaches, or contingencies remains unclear. Yet, typifying the pathways of change and reflecting on them is the first step towards comprehensive evaluation of when and which pathways and interactions to promote.

Caracterización de la Visión de los Profesionales de la Conservación sobre el Papel de la Educación Resumen La educación es una herramienta consolidada para mejorar las relaciones entre la humanidad y el medio ambiente, a pesar de la falta de evidencia empírica que respalde su uso. Usamos teorías del cambio para desentrañar las suposiciones sobre el papel de la educación en la conservación. Entrevistamos a profesionales de 15 organizaciones de conservación en Madagascar para caracterizar las vías implícitas de cambio y para analizar si las vías emergentes reflejan los avances teóricos. Cinco vías fueron impulsoras del cambio: el incremento del conocimiento, el cambio de la conexión emocional y el cambio de las prácticas culturales tradicionales, el fomento al liderazgo, la diversificación de resultados, y la influencia sobre la comunidad y la sociedad. Estas vías reflejan las teorías socio-psicológicas existentes sobre el aprendizaje y el cambio de comportamiento. La mayoría de las organizaciones de los entrevistados tuvo una vía predominante que con frecuencia estuvo combinada con los elementos de otras vías. La mayoría de las vías careció de estrategias fundamentadas culturalmente. Nuestra investigación reveló las suposiciones sobre el papel de la educación en la conservación e indica que las organizaciones tenían ideas diferentes sobre cómo ocurren los cambios. La diversidad de prácticas refleja la complejidad de los factores que influyen en el comportamiento. Todavía no está claro si esta diversidad es causada por el contexto sociocultural local, la interacción con otras estrategias de conservación o con las eventualidades. Aun así, caracterizar las vías de cambio y reflexionar sobre ellas es el primer paso hacia la evaluación integral de cuándo y cuáles vías e interacciones se deben promover.

"We are not stray leaves blowing about in the wind": exploring the impact of Family Wellbeing empowerment research, 1998-2021.

BACKGROUND: An Aboriginal-developed empowerment and social and emotional wellbeing program, known as Family Wellbeing (FWB), has been found to strengthen the protective factors that help Indigenous Australians to deal with the legacy of colonisation and intergenerational trauma. This article reviews the research that has accompanied the implementation of the program, over a 23 year period. The aim is to assess the long-term impact of FWB research and identify the key enablers of research impact and the limitations of the impact assessment exercise. This will inform more comprehensive monitoring of research impact into the future. METHODS: To assess impact, the study took an implementation science approach, incorporating theory of change and service utilisation frameworks, to create a logic model underpinned by Indigenous research principles. A research impact narrative was developed based on mixed methods analysis of publicly available data on: 1) FWB program participation; 2) research program funding; 3) program outcome evaluation (nine studies); and 4) accounts of research utilisation (seven studies). RESULTS: Starting from a need for research on empowerment identified by research users, an investment of $2.3 million in research activities over 23 years produced a range of research outputs that evidenced social and emotional wellbeing benefits arising from participation in the FWB program. Accounts of research utilisation confirmed the role of research outputs in educating participants about the program, and thus, facilitating more demand (and funding acquisition) for FWB. Overall research contributed to 5,405 recorded participants accessing the intervention. The key enablers of research impact were; 1) the research was user- and community-driven; 2) a long-term mutually beneficial partnership between research users and researchers; 3) the creation of a body of knowledge that demonstrated the impact of the FWB intervention via different research methods; 4) the universality of the FWB approach which led to widespread application. CONCLUSIONS: The FWB research impact exercise reinforced the view that assessing research impact is best approached as a "wicked problem" for which there are no easy fixes. It requires flexible, open-ended, collaborative learning-by-doing approaches to build the evidence base over time. Steps and approaches that research groups might take to build the research impact knowledge base within their disciplines are discussed.

Using a theory of change in monitoring, evaluating and steering scale-up of a district-level health management strengthening intervention in Ghana, Malawi, and Uganda - lessons from the PERFORM2Scale consortium.

BACKGROUND: Since 2017, PERFORM2Scale, a research consortium with partners from seven countries in Africa and Europe, has steered the implementation and scale-up of a district-level health management strengthening intervention in Ghana, Malawi and Uganda. This article presents PERFORM2Scale's theory of change (ToC) and reflections upon and adaptations of the ToC over time. The article aims to contribute to understanding the benefits and challenges of using a ToC-based approach for monitoring and evaluating the scale-up of health system strengthening interventions, because there is limited documentation of this in the literature. METHODS: The consortium held annual ToC reflections that entailed multiple participatory methods, including individual scoring exercises, country and consortium-wide group discussions and visualizations. The reflections were captured in detailed annual reports, on which this article is based. RESULTS: The PERFORM2Scale ToC describes how the management strengthening intervention, which targets district health management teams, was expected to improve health workforce performance and service delivery at scale, and which assumptions were instrumental to track over time. The annual ToC reflections proved valuable in gaining a nuanced understanding of how change did (and did not) happen. This helped in strategizing on actions to further steer the scale-up the intervention. It also led to adaptations of the ToC over time. Based on the annual reflections, these actions and adaptations related to: assessing the scalability of the intervention, documentation and dissemination of evidence about the effects of the intervention, understanding power relationships between key stakeholders, the importance of developing and monitoring a scale-up strategy and identification of opportunities to integrate (parts of) the intervention into existing structures and strategies. CONCLUSIONS: PERFORM2Scale's experience provides lessons for using ToCs to monitor and evaluate the scale-up of health system strengthening interventions. ToCs can help in establishing a common vision on intervention scale-up. ToC-based approaches should include a variety of stakeholders and require their continued commitment to reflection and learning on intervention implementation and scale-up. ToC-based approaches can help in adapting interventions as well as scale-up processes to be in tune with contextual changes and stakeholders involved, to potentially increase chances for successful scale-up.

Identifying key risks to the performance of privately protected areas (PPAs) through theory of change (ToC).

Privately protected areas (PPAs) are internationally considered to be important policy implementation instruments to augment and strengthen protected area networks. However, there has been limited reflection on the performance of PPAs over time. This paper aims to identify key risks to the performance of PPAs as policy implementation instruments through the application of Theory of Change (ToC). Identifying and understanding these risks are important to allow for the evaluation and monitoring of PPA performance. The ToC method was applied to a specific PPA policy instrument namely, private nature reserves (PNRs) in the South African context. The research results produced 29 key assumptions translated into 29 key risks. These risk are critically discussed against existing South African and international literature. To test and refine the risks further it is recommended that they be applied to PPA case studies in different contexts.