Kenya's Experience: Factors Enabling and Impeding the COVID-19 Response.

This case study describes the country-level response to the COVID-19 pandemic in Kenya between February 2020 and May 2021. We organize the presentation of COVID-19 response strategies across the five stages of (a) engagement, (b) assessment, (c) planning, (d) action/implementation, and (e) evaluation. We describe the participatory monitoring and evaluation (M&E) process implemented in collaboration with the WHO Regional Office for Africa Monitoring and Evaluation Team. The M&E system was used to organize and make sense of emerging data regarding specific response activities and changing COVID incidence. We share the results of that collaborative sensemaking, with particular attention to our analysis of the factors that facilitated and those that impeded our pandemic response. We conclude with lessons learned and practical implications from Kenya's experience to help guide future country-level responses to rapidly changing public health crises.

Evaluation of the Grow Your Groceries Home Gardening Program in Chicago, Illinois.

COVID-19 exacerbated existing disparities in food security in Chicago. Home gardening can improve food security but there are often barriers to participation and the benefits are understudied. Chicago Grows Food (CGF) formed in 2020 to address food insecurity during COVID-19, and created the Grow Your Groceries (GYG) program to provide home gardening kits to families at risk of food insecurity in Chicago. A participatory program evaluation was conducted to better understand the experiences of and benefits to individuals participating in GYG. Program participants shared feedback via focus groups (n = 6) and surveys (n = 72). Qualitative data were analyzed using an iterative coding process. Quantitative data were analyzed using descriptive statistics. Most participants reported confidence in using a grow kit to grow food, increased healthy food consumption, easier access to healthy food, and high likelihood of growing food again. Additionally, participants described increased connections within their communities, increased interaction with their family, and personal growth as benefits of the program. These results demonstrate the benefits of a novel home gardening program that uses fabric grow bags to address food insecurity. A larger scale program evaluation is necessary to better understand the impacts of participating in this home gardening program.

Participatory Monitoring and Evaluation of the COVID-19 Response in the Africa Region.

As the COVID-19 pandemic swept throughout the world, it created a demand for information to help understand the public health response and its effects. Limited capacity to see and interpret data-"sensemaking" with measures of progress-affects the use of data for quality improvement. The World Health Organization Regional Office for Africa (WHO AFRO) supported partners from the Member States in using a participatory monitoring and evaluation system to document and systematically reflect on the COVID-19 response at the country level. The WHO AFRO's COVID-19 Response Monitoring and Evaluation (M&E) team captured and communicated response activities based on available reports from 35 of the 47 member countries. By reviewing reports and communications, the M&E team documented nearly 8,000 COVID-19 response activities during the study period (January 2020 through July 2021). A "sensemaking" protocol was used to support country partners in identifying factors associated with increases or decreases in both new cases and response activities. This report describes this participatory M&E approach and process of shared sensemaking. We illustrate with a country-level case study of the COVID-19 response in the Africa Region.

Equity Lens Protocol: Reflecting on Harms From the COVID-19 Response and Mitigation Strategies Implemented in a Local Public Health System.

This report describes an Equity Lens Protocol and its use to guide partners' systematic reflection on harms and mitigation strategies of the COVID-19 response in a local public health system. This process evaluation tool is based on the Guidance document for assuring an equitable response to COVID-19 prepared by the Pan American Health Organization. We used a participatory approach to engage public health partners in systematically reflecting on harms, mitigation strategies, and lessons learned and implications for practice. Outputs from using this tool included identified: (a) specific harms (e.g., loss of income and challenges to learning) related to particular COVID-19 response measures (e.g., home confinement and school closure) and (b) mitigation strategies implemented to reduce harms. In response to the protocol's guiding questions, partners also identified lessons learned and practice recommendations for strengthening equity work in public health responses (e.g., an equitable response requires an investment in people, structures, and relationships before a crisis). This report-and accompanying protocol-illustrates use of a practical method for systematic reflection on public health responses through an equity lens.

Community-based intervention improves abortion knowledge and reduces abortion stigma among women in Oromia, Ethiopia: a quasi-experimental mixed methods evaluation.

BACKGROUND: Since liberalization of the Ethiopian abortion law, there have been significant improvements in the availability and utilization of facility-based abortion services in the country. However, nearly half of abortions still take place outside of health facilities, where the quality of procedures remains unknown. Abortion stigma is one reason that unsafe abortion persists. This study aims to evaluate the effect of community interventions conducted from 2016 to 2019 on the level and manifestation of abortion stigma and knowledge in a community in Oromia region, Ethiopia. METHODS: The study is a quasi-experimental mixed methods evaluation including intervention and comparison communities. Two cross-sectional structured household surveys with independent samples, participatory evaluation wheels, and participatory impact diagrams were conducted with women of reproductive age (15-49) living in the communities. The baseline was conducted in 2016 and the endline in 2019. Difference-in-differences analysis was used to estimate the effect of the intervention on abortion knowledge and Stigmatizing Attitudes, Beliefs, and Actions Scale (SABAS) scores in the intervention community. RESULTS: One thousand five hundred fifty-five women participated in the household survey and 28 women participated in participatory evaluation meetings. Over one-third (37%) of women surveyed in the intervention community were exposed to the intervention activities. Knowledge of one or more indications of legal abortion increased from 21 to 85% in the intervention community, compared to an increase from 30 to 57% in the comparison. Mean SABAS scores decreased by 9.3 points in the intervention community and increased by 5.3 points in the comparison community. Differences-in-differences models indicate that exposure to the intervention resulted in decreased stigma scores (coefficient = - 9.33, p < 0.001) and increased knowledge (coefficient = 0.26, p < 0.001). CONCLUSIONS: This is one of the first studies to measure changes in community-level abortion stigma and knowledge over time in Ethiopia using a mixed method, quasi-experimental design. The results indicate that the community-based intervention improved abortion knowledge and reduced abortion stigma.

Engaging young people within a collaborative knowledge mobilization network: Development and evaluation.

BACKGROUND: It is critical that mental health systems place a focus on prevention and early intervention focused on young people while integrating youth voice to guide priority directions. OBJECTIVE: This study was designed to better understand how youth advisories can be utilized to influence strategic directions within integrated knowledge mobilization networks operating within the youth mental health system. DESIGN: To support this objective, we reviewed the detailed stages of development in establishing a youth advisory within a national network designed to support the integration of youth services. We also engaged the advisory in a participatory evaluation process that examined the extent to which the network had created processes to include youth voice in decision-making. RESULTS: Results from the surveys identified moderate to high levels of individual engagement as well as strong development of processes and procedures that support the inclusion of youth voice across the network. DISCUSSION: Major successes and challenges are presented and discussed with respect to the development of the advisory. The findings are useful for youth advocates and adult allies working to support youth engagement (YE) in knowledge mobilization to enhance the mental health services system. This study also contributes to research and evaluation efforts examining YE and represents an exemplar methodology for evaluating YE efforts at the system level. PATIENT OR PUBLIC CONTRIBUTION: Young people as mental health service users and youth mental health advocates were involved in the design, data collection, analysis and interpretation of the data as well as the preparation of this manuscript.

Empowering Indigenous Communities Through a Participatory, Culturally Responsive Evaluation of a Federal Program for Older Americans.

This article describes our experience of conducting a 5-year, culturally responsive evaluation of a federal program with Indigenous communities. It describes how we adapted tenets from "participatory evaluation models" to ensure cultural relevance and empowerment. We provide recommendations for evaluators engaged in similar efforts. The evaluation included stakeholder engagement through a Steering Committee and an Evaluation Working Group in designing and implementing the evaluation. That engagement facilitated attention to Indigenous cultural values in developing a program logic model and medicine wheel and in gathering local perspectives through storytelling to facilitate understanding of community traditions. Our ongoing assessment of program grantees' needs shaped our approach to evaluation capacity building and development of a diverse array of experiential learning opportunities and user-friendly tools and resources. We present practical strategies from lessons learned during the evaluation design and implementation phases of our project that might be useful for other evaluators.

A nationwide participatory programme to measure person-centred hospital care in Italy: Results and implications for continuous improvement.

BACKGROUND: Patient-centredness has been targeted by the Italian government as a key theme for the future development of health services. OBJECTIVE: Measuring patient-centred health services in partnership with citizens, health professionals and decision makers. DESIGN: National participatory survey in a large test set of hospitals at national level. SETTING AND PARTICIPANTS: A total of 387 hospital visits conducted in 16 Italian regions by over 1,500 citizens and health professionals during 2017-2018. MAIN VARIABLES AND OUTCOME MEASURES: An ad hoc checklist was used to assess person-centredness in hospital care through 243 items, grouped in 4 main areas, 12 sub-areas and 29 person-centred criteria (scored 0-10). GEE linear multivariate regression was used to explore the relation between hospital characteristics and person-centredness. RESULTS: Person-centred scores were moderately high, with substantial variation overall (median score: 7.0, range: 3.2-9.5) and by area (Care Processes: 6.8, 2.0-9.8; Access: 7.4, 2.7-9.7; Transparency: 6.7, 3.4-9.5 and Relationship: 7.3, 0.8-10.0). Multivariate regression found higher scores for increasing volumes of activity (quartile increase: +0.21; 95% CI: 0.13, 0.29) and lower scores in the south and islands (-1.03; -1.62,-0.45). DISCUSSION: The checklist has been applied successfully by over 1,500 collaborators who assessed hospitals in 16 distinct Regions and Autonomous Provinces of Italy. Despite an overall positive mark, all scores were highly variable by location and hospital characteristics. CONCLUSION AND PATIENT OR PUBLIC CONTRIBUTION: A national participatory programme to improve patient-centredness in Italian hospitals highlighted critical areas with the direct input of citizens.

Participatory Monitoring and Evaluation of the COVID-19 Response in a Local Public Health System.

The coronavirus disease 2019 (COVID-19) pandemic tested the capacity of local health systems to understand and respond to changing conditions. Although data on new cases of COVID-19 were widely shared in communities, there was less information on the multisector response activities and factors associated with implementation. To address this gap, this empirical case study examined (a) the pattern of implementation of COVID-19 response activities and (b) the factors and critical events associated with both the pattern of new cases and the implementation of the local COVID-19 response. We used a participatory monitoring and evaluation system to capture, code, characterize, and communicate 580 COVID-19 response activities implemented in the city of Lawrence and Douglas County, Kansas. Collaboration across sectors including public health, medical services, city/county government, businesses, social services, public schools, and universities enabled the local public health system's response effort. Documentation results showed the varying pattern of new COVID-19 cases and response activities over time and the factors identified as enabling or impeding the response and related new cases. Similar participatory monitoring and evaluation methods can be used by local health systems to help understand and respond to the changing conditions of COVID-19 response and recovery.

On-farm performance and farmers' participatory assessment of new stress-tolerant maize hybrids in Eastern Africa.

The development and deployment of high-yielding stress tolerant maize hybrids are important components of the efforts to increase maize productivity in eastern Africa. This study was conducted to: i) evaluate selected, stress-tolerant maize hybrids under farmers' conditions; ii) identify farmers' selection criteria in selecting maize hybrids; and iii) have farmers evaluate the new varieties according to those criteria. Two sets of trials, one with 12 early-to-intermediate maturing and the other with 13 intermediate-to-late maturing hybrids, improved for tolerance to multiple stresses common in farmers' fields in eastern Africa (drought, northern corn leaf blight, gray leaf spot, common rust, maize streak virus), were evaluated on-farm under smallholder farmers' conditions in a total of 42 and 40 environments (site-year-management combinations), respectively, across Kenya, Uganda, Tanzania and Rwanda in 2016 and 2017. Farmer-participatory variety evaluation was conducted at 27 sites in Kenya and Rwanda, with a total of 2025 participating farmers. Differential performance of the hybrids was observed under low-yielding (<3 t ha-1) and high-yielding (>3 t ha-1) environments. The new stress-tolerant maize hybrids had a much better grain-yield performance than the best commercial checks under smallholder farmer growing environments but had a comparable grain-yield performance under optimal conditions. These hybrids also showed better grain-yield stability across the testing environments, providing an evidence for the success of the maize-breeding approach. In addition, the new stress- tolerant varieties outperformed the internal genetic checks, indicating genetic gain under farmers' conditions. Farmers gave high importance to grain yield in both farmer-stated preferences (through scores) and farmer-revealed preferences of criteria (revealed by regressing the overall scores on the scores for the individual criteria). The top-yielding hybrids in both maturity groups also received the farmers' highest overall scores. Farmers ranked yield, early maturity, cob size and number of cobs as the most important traits for variety preference. The criteria for the different hybrids did not differ between men and women farmers. Farmers gave priority to many different traits in addition to grain yield, but this may not be applicable across all maize-growing regions. Farmer-stated importance of the different criteria, however, were quite different from farmer- revealed importance. Further, there were significant differences between men and women in the revealed-importance of the criteria. We conclude that incorporating farmers' selection criteria in the stage-gate advancement process of new hybrids by the breeders is useful under the changing maize-growing environments in sub-Saharan Africa, and recommended to increase the turnover of new maize hybrids.

The Fortune of the Commons: Participatory Evaluation of Small-Scale Fisheries in the Brazilian Amazon.

This paper applies a participatory approach in evaluating small-scale fisheries, focusing on the Arapaima gigas fishery in the Brazilian Amazon. The evaluation uses the social-ecological system (SES) framework, adopted to explain the conditions needed for sustainability and user cooperation in natural resources management, as a more suitable alternative to the 'blueprint' or 'panaceas' approaches, based only on property rights or governmental intervention. However, managers and users often do not have the necessary information compiled and available for a specific SES while some actions need to be taken immediately. Thus, consensus and negotiation among stakeholders about SES variables may be useful to evaluate system performance and indicate actions to promote sustainability. In the case study, using a consensus-building model, we found that arapaima SES leads to sustainability and is far from being a case of 'tragedy of the commons.' More investments in suitable monitoring and enforcement for adaptive management are recommended. Adopting an SES framework based on stakeholders' prospects may be useful until complete interdisciplinary studies become available so as to seek of sustainability in the long term.

The implementation of a participatory manuscript development process with Native American tribal awardees as part of the CDC Communities Putting Prevention to Work initiative: challenges and opportunities.

OBJECTIVE: In 2009, the Centers for Disease Control and Prevention funded 50 communities, including three tribal awardees, to implement environmental approaches to address obesity and smoking through the Communities Putting Prevention to Work initiative. The tribes were among the selected awardees offered training support for analyzing, writing, and publishing their findings. This article describes the process of translating the workshops, guided by a participatory framework, for implementation with the tribes. METHODS: Nine participants from three tribes attended the workshops in Decatur, Georgia, in August and October of 2012: 1) a one-day pre-conference workshop focused on integrating both Indigenous and academic evaluation methods; 2) a 4 day data analysis workshop; and 3) a 5 day scientific writing workshop. Participants were provided with technical assistance following the workshops. RESULTS: Participants viewed the workshops positively and have continued to develop their manuscripts. To date one tribal awardee has submitted their manuscript for publication. CONCLUSION: The participatory manuscript development process described here is the first of its kind outlining a pathway for tribal community health practitioners to translate and publish their work. Further development of this process could increase the number of community-developed manuscripts, thereby advancing the field of translational intervention science and leading to improved health equity.

Implementation and evaluation of the chronic-disease self-management program among Chinese immigrant older adults in the U.S.

This article describes the implementation and evaluation of the chronic-disease self-management (CDSM) program, developed by Stanford University, among Chinese older adults in a metro area of a large Southeastern City of the U.S. The method of Practical Participatory Evaluation through an academic-community partnership between university researchers and local Chinese communities was used to develop the program and assess its applicability in the population. Results suggested that language proficiency, communication, social network and culture of the population were the most influential factors for U.S. Chinese immigrants to attend the CDSM program. The program increased participants' knowledge, skills and confidence in CDSM, whereas its capability in addressing culture differences needed improvement. Knowledge learned in this project was instrumental in implementing similar projects among immigrants.

Using Ripple Effects Mapping to understand the longer-term impacts of delivering a dance programme for older adults on dance artists.

BACKGROUND: The dance workforce plays a central role in delivering arts and health programmes yet there is little exploration of how programme delivery impacts dance artists in a professional or personal capacity. This study explored the experiences of dance artists delivering Dance On, which engages inactive older people 55yrs+. METHODS: Ripple Effects Mapping was used to explore the short- and long term experiences and practices of dance artists delivering a dance programme. FINDINGS: Two ripples were developed 1) Becoming a specialist 2) Connecting with communities. These ripples highlighted the strengths of the sustained nature of the programme and emphasised the need for ongoing support from employing organisations, communities, and dance artist peers. CONCLUSION: This study showcases the central role dance artists play in upholding the outcomes we observe in arts and health work - their role, expertise, and commitment to programmes should be further illuminated and supported through ongoing discourse about their practice.

Participatory training evaluation: Steps from the Center for Native American Health Native-CHART symposium.

This paper presents a case example of the Native-CHART Training Evaluation and describes the process of planning and administering a paper evaluation during the Native-CHART symposium in November 2019 led by the Center for Native American Health (CNAH) and an external evaluator. Training evaluation methodologies and the data collection instrument were grounded in the Health Belief Model (HBM) where health-related chronic disease and risk factor knowledge translates to perceived susceptibility, benefits, barriers, and self-efficacy. Kirkpatrick's Four-level Training Evaluation Model explored learning, reaction, behaviors, and results. The evaluation aims centered around the following questions: 1)Who attended the symposium, and why did they attend? 2)What knowledge did participants gain at the symposium? 3)Will attendees change their behaviors as a result of attending the symposium? 4) What parts of the symposium were most valuable? And 5) How can the symposium be improved? Data collected at the symposium answered these questions. After the Native-CHART symposium, CNAH staff and the external evaluator met to reflect on the steps necessary to plan and implement a participatory training evaluation. From these discussions, eight steps emerged. This paper presents these steps along with recommendations for future work. Participatory and collaborative approaches in training evaluation and the steps included in this case example may be useful to evaluators, communities, and programs working on designing and evaluating various trainings with Tribal populations.

Using photovoice as a participatory evaluation tool in Kaiser Permanente's Community Health Initiative.

Photovoice is a community-based participatory research method that provides participants who traditionally have little voice in community policy decisions, with training in photography, ethics, critical dialogue, photo captioning, and policy advocacy. Photovoice has been used primarily as a needs assessment and advocacy tool and only rarely as a pre-/postintervention evaluation method. This article describes the use of Photovoice as a participatory evaluation method in the Community Health Initiative, a 6-year, multisite community-based obesity prevention initiative, sponsored by Kaiser Permanente. Fifty community participants (including six youth) from six Community Health Initiative communities used photos and captions to identify, from their perspective, the most significant accomplishments from the initiative at both baseline and follow-up. Accomplishments identified included increased access to fresh/healthy food in local neighborhoods; policy changes supporting a "healthy eating, active living" community; increased access to physical activity; changes to the built environment creating increased neighborhood walkability/safety; and leadership development.

Evaluating a moving target: Using Practical Participatory Evaluation (P-PE) in hospital settings.

This article describes lessons learned about implementing evaluations in hospital settings. In order to overcome the methodological dilemmas inherent in this environment, we used a practical participatory evaluation strategy to engage as many stakeholders as possible in the process of evaluating a clinical demonstration project. Demonstration projects, in this context, push the envelope about what is known about effectiveness in novel settings, and turnover of staff and patient populations can present challenges to gathering optimal data. By using P-PE, we built capacity in the environment while expanding possibilities for data collection. Suggestions are made based on our experience.

Participatory evaluation of municipal obesity prevention clubs in Tehran city: Strengths, challenges, and future direction.

Background and aim: Community-based initiatives are important for obesity prevention. This study aimed to evaluate the activities of municipal obesity prevention clubs (OBCs) in Tehran, Iran, using a participatory approach. Methods: The evaluation team was formed, and the members identified the OBC's strengths, and challenges and provided suggestions for change through a participatory workshop, observation, focus group discussions, reviewing relevant documents (n = 97), and 35 interviews with involved stakeholders. The MAXQDA software was used for data analysis. Results: An empowerment training program for volunteers was identified as one of the strengths of OBCs. Despite the obesity prevention efforts of OBCs through public exercise sessions, healthy food festivals, and educational sessions, several challenges were identified that hinder participation in OBCs. These challenges included poor marketing strategies, poor training approaches in participatory planning, insufficient motivational support for volunteers, low perceived recognition of volunteers by the community, volunteers' low food and nutrition literacy, poor educational services in the communities, and limited funding for health promotion activities. Conclusion: Weaknesses in all stages of community participation, including information, consultation, collaboration, and empowerment, in OBCs were detected. Facilitating a more enabling environment for informing and involving citizens, expanding neighborhood social capital, and involving health volunteers, academia, and all potential governmental sectors to collaborate for obesity prevention is recommended.

How are Research for Development Programmes Implementing and Evaluating Equitable Partnerships to Address Power Asymmetries?

The complexity of issues addressed by research for development (R4D) requires collaborations between partners from a range of disciplines and cultural contexts. Power asymmetries within such partnerships may obstruct the fair distribution of resources, responsibilities and benefits across all partners. This paper presents a cross-case analysis of five R4D partnership evaluations, their methods and how they unearthed and addressed power asymmetries. It contributes to the field of R4D partnership evaluations by detailing approaches and methods employed to evaluate these partnerships. Theory-based evaluations deepened understandings of how equitable partnerships contribute to R4D generating impact and centring the relational side of R4D. Participatory approaches that involved all partners in developing and evaluating partnership principles ensured contextually appropriate definitions and a focus on what partners value. Supplementary Information: The online version contains supplementary material available at 10.1057/s41287-023-00578-w.

Finally, centring reflexivity within a learning oriented approach ensured that partnership evaluation findings were used to adapt and improve the way R4D programmes operate. La complexité des enjeux abordés par la recherche pour le développement (R4D) nécessite des collaborations entre des partenaires de disciplines et de contextes culturels variés. Les asymétries de pouvoir au sein d'un tel partenariat peuvent entraver la répartition équitable des ressources, des responsabilités et des avantages entre tous les partenaires. Cet article présente une analyse croisée de cinq évaluations de partenariats R4D, leurs méthodes et la manière dont elles ont mis au jour et traité les asymétries de pouvoir. Cet article contribue au domaine des évaluations de partenariats R4D en décrivant de façon détaillée les approches et les méthodes employées pour évaluer ces partenariats. Les évaluations basées sur la théorie ont permis d'approfondir la compréhension de la manière dont les partenariats équitables contribuent à l'impact de la R4D et à mettre au centre l'aspect relationnel de la R4D. Les approches participatives impliquant tous les partenaires dans l'élaboration et l'évaluation des principes du partenariat ont permis de garantir des définitions adaptées au contexte et de faire en sorte que les évaluations se penchent sur ce qui est jugé important pour tous les partenaires. Enfin, le fait de mettre la réflexivité au cœur de l'analyse dans une approche axée sur l'apprentissage a permis d'utiliser les résultats de l'évaluation du partenariat pour adapter et améliorer le fonctionnement des programmes de R4D.