Characterizing the implementation of performance management interventions in a primary health care system: a case study of the Salud Mesoamerica Initiative in El Salvador.

Performance management (PM) reforms have been introduced in health systems worldwide to improve accountability, transparency and learning. However, gaps in evidence exist regarding the ways in which PM contributes to organizational-level outcomes. Between 2015 and 2017, the government of El Salvador and the Salud Mesoamerica Initiative (SMI) introduced team-based PM interventions in the country's primary health care (PHC) system including target setting, performance measurement, provision of feedback and in-kind incentives. The programme's evaluation showed widespread improvements in performance for community outreach and service timeliness, quality and utilization. The current study characterizes how the implementation of team-based PM interventions by SMI implementers contributed to PHC system performance improvements. We used a descriptive, single-case study design informed by a programme theory (PT). Data sources included qualitative in-depth interviews and SMI programme documents. We interviewed the members of four PHC teams (n = 13), Ministry of Health (MOH) decision makers (n = 8) and SMI officials (n = 6). Coded data were summarized, and thematic analysis was employed to identify broader categories and patterns. The outcomes chain in the PT was refined based on empirical findings that revealed the convergence of two processes: (1) increased social interactions and relationships among implementers that enhanced communication and created opportunities for social learning and (2) cyclical performance monitoring that generated novel flows of information. These processes contributed to emergent outcomes including the uptake of performance information, altruistic behaviours in service delivery and organizational learning. Through time, the cyclical nature of PM appears to have led to the spread of these behaviours beyond the teams studied here, thus contributing to system-wide effects. Findings illustrate the social nature of implementation processes and describe plausible pathways through which lower-order implementation programme effects can contribute to higher-order changes in system performance.

Characterising performance information use in the primary healthcare systems of El Salvador, Lebanon and Malawi: multiple qualitative case study protocol.

INTRODUCTION: Governments in low-income and middle-income countries (LMICs) and official development assistance agencies use a variety of performance measurement and management approaches to improve the performance of healthcare systems. The effectiveness of such approaches is contingent on the extent to which managers and care providers use performance information. To date, major knowledge gaps exist about the contextual factors that contribute, or not, to performance information use by primary healthcare (PHC) decision-makers in LMICs. This study will address three research questions: (1) How do decision-makers use performance information, and for what purposes? (2) What are the contextual factors that influence the use or non-use of performance information? and (3) What are the proximal outcomes reported by PHC decision-makers from performance information use? METHODS AND ANALYSIS: We present the protocol of a theory-driven, qualitative study with a multiple case study design to be conducted in El Salvador, Lebanon and Malawi.Data sources include semi structured in-depth interviews and document review. Interviews will be conducted with approximately 60 respondents including PHC system decision-makers and providers. We follow an interdisciplinary theoretical framework that draws on health policy and systems research, public administration, organisational science and health service research. Data will be analysed using thematic analysis to explore how respondents use performance information or not, and for what purposes as well as barriers and facilitators of use. ETHICS AND DISSEMINATION: The ethical boards of the participating universities approved the protocol presented here. Study results will be disseminated through peer-reviewed journals and global health conferences.

Improving primary health care delivery in Bihar, India: Learning from piloting and statewide scale-up of Ananya.

In 2010, the Bill and Melinda Gates Foundation (BMGF) partnered with the Government of Bihar (GoB), India to launch the Ananya program to improve reproductive, maternal, newborn and child health and nutrition (RMNCHN) outcomes. The program sought to address supply- and demand-side barriers to the adoption, coverage, quality, equity and health impact of select RMNCHN interventions. Approaches included strengthening frontline worker service delivery; social and behavior change communications; layering of health, nutrition and sanitation into women's self-help groups (SHGs); and quality improvement in maternal and newborn care at primary health care facilities. Ananya program interventions were piloted in approximately 28 million population in eight innovation districts from 2011-2013, and then beginning in 2014, were scaled up by the GoB across the rest of the state's population of 104 million. A Bihar Technical Support Program provided techno-managerial support to governmental Health as well as Integrated Child Development Services, and the JEEViKA Technical Support Program supported health layering and scale-up of the GoB's SHG program. The level of support at the block level during statewide scale-up in 2014 onwards was approximately one-fourth that provided in the pilot phase of Ananya in 2011-2013. This paper - the first manuscript in an 11-manuscript and 2-viewpoint collection on Learning from Ananya: Lessons for primary health care performance improvement - seeks to provide a broad description of Ananya and subsequent statewide adaptation and scale-up, and capture the background and context, key objectives, interventions, delivery approaches and evaluation methods of this expansive program. Subsequent papers in this collection focus on specific intervention delivery platforms. For the analyses in this series, Stanford University held key informant interviews and worked with the technical support and evaluation grantees of the Ananya program, as well as leadership from the India Country Office of the BMGF, to analyse and synthesise data from multiple sources. Capturing lessons from the Ananya pilot program and statewide scale-up will assist program managers and policymakers to more effectively design and implement RMNCHN programs at scale through technical assistance to governments.

Evidence gap map of performance measurement and management in primary healthcare systems in low-income and middle-income countries.

INTRODUCTION: We mapped available evidence on performance measurement and management (PMM) strategies in primary healthcare (PHC) systems of low-income and middle-income countries (LMICs). Widely used, their effectiveness remains inconclusive. This evidence gap map characterises existing research and evidence gaps. METHODS: Systematic mapping of performance measurement and management research in LMICs from 2000 to mid-2018; literature searches of seven academic databases and institutional repositories of impact evaluations and systematic reviews. Using a combination of manual screening and machine learning, four reviewers appraised 38 088 titles and abstracts, and extracted metadata from 137 impact evaluations and 18 systematic reviews that met the inclusion criteria. The resulting visual representation of the evidence base was uploaded to a web-based platform. RESULTS: Since 2000, the number of studies has increased; the first systematic reviews were completed in 2010. Two-thirds of the studies were conducted in sub-Saharan Africa and South Asia. Randomised controlled trials were the most frequently used study design. The evidence is concentrated in two types of PMM strategies: implementation strategies (in-service training, continuing education, supervision) and performance-based financing. Major gaps exist in accountability arrangements particularly the use of audit and feedback. The least studied types of outcomes were unintended effects, harm and social equity. CONCLUSIONS: The evidence is clustered around interventions that are unlikely to achieve transformational change in health outcomes. The gaps identified suggest that routinely used PMM strategies are implemented without sufficient knowledge of their effects. Future efforts at redesigning PHC systems need to be informed by evidence on the most effective approaches for using PMM strategies.

Effects of team-based goals and non-monetary incentives on front-line health worker performance and maternal health behaviours: a cluster randomised controlled trial in Bihar, India.

INTRODUCTION: We evaluated the impact of a 'Team-Based Goals and Incentives' (TBGI) intervention in Bihar, India, designed to improve front-line (community health) worker (FLW) performance and health-promoting behaviours related to reproductive, maternal, newborn and child health and nutrition. METHODS: This study used a cluster randomised controlled trial design and difference-in-difference analyses of improvements in maternal health-related behaviours related to the intervention's team-based goals (primary), and interactions of FLWs with each other and with maternal beneficiaries (secondary). Evaluation participants included approximately 1300 FLWs and 3600 mothers at baseline (May to June 2012) and after 2.5 years of implementation (November to December 2014) who had delivered an infant in the previous year. RESULTS: The TBGI intervention resulted in significant increases in the frequency of antenatal home visits (15 absolute percentage points (PP), p=0.03) and receipt of iron-folic acid (IFA) tablets (7 PP, p=0.02), but non-significant changes in other health behaviours related to the trial's goals. Improvements were seen in selected attitudes related to coordination and teamwork among FLWs, and in the provision of advice to beneficiaries (ranging from 8 to 14 PP) related to IFA, cord care, breast feeding, complementary feeding and family planning. CONCLUSION: Results suggest that combining an integrated set of team-based coverage goals and targets, small non-cash incentives for teams who meet targets and team building to motivate FLWs resulted in improvements in FLW coordination and teamwork, and in the quality and quantity of FLW-beneficiary interactions. These improvements represent programmatically meaningful steps towards improving health behaviours and outcomes. TRIAL REGISTRATION NUMBER: NCT03406221.

Characterizing performance improvement in primary care systems in Mesoamerica: A realist evaluation protocol.

Background. Improving performance of primary care systems in low- and middle-income countries (LMICs) may be a necessary condition for achievement of universal health coverage in the age of Sustainable Development Goals. The Salud Mesoamerica Initiative (SMI), a large-scale, multi-country program that uses supply-side financial incentives directed at the central-level of governments, and continuous, external evaluation of public, health sector performance to induce improvements in primary care performance in eight LMICs. This study protocol seeks to explain whether and how these interventions generate program effects in El Salvador and Honduras. Methods. This study presents the protocol for a study that uses a realist evaluation approach to develop a preliminary program theory that hypothesizes the interactions between context, interventions and the mechanisms that trigger outcomes. The program theory was completed through a scoping review of relevant empirical, peer-reviewed and grey literature; a sense-making workshop with program stakeholders; and content analysis of key SMI documents. The study will use a multiple case-study design with embedded units with contrasting cases. We define as a case the two primary care systems of Honduras and El Salvador, each with different context characteristics. Data will be collected through in-depth interviews with program actors and stakeholders, documentary review, and non-participatory observation. Data analysis will use inductive and deductive approaches to identify causal patterns organized as 'context, mechanism, outcome' configurations. The findings will be triangulated with existing secondary, qualitative and quantitative data sources, and contrasted against relevant theoretical literature. The study will end with a refined program theory. Findings will be published following the guidelines generated by the Realist and Meta-narrative Evidence Syntheses study (RAMESES II). This study will be performed contemporaneously with SMI's mid-term stage of implementation. Of the methods described, the preliminary program theory has been completed. Data collection, analysis and synthesis remain to be completed.

Evidence gap map of performance measurement and management in primary care delivery systems in low- and middle-income countries - Study protocol.

Background . For the last two decades there has been growing interest in governmental and global health stakeholders about the role that performance measurement and management systems can play for the production of high-quality and safely delivered primary care services. Despite recognition and interest, the gaps in evidence in this field of research and practice in low- and middle-income countries remain poorly characterized. This study will develop an evidence gap map in the area of performance management in primary care delivery systems in low- and middle-income countries. Methods. The evidence gap map will follow the methodology developed by 3Ie, the International Initiative for Impact Evaluation, to systematically map evidence and research gaps. The process starts with the development of the scope by creating an evidence-informed framework that helps identify the interventions and outcomes of relevance as well as help define inclusion and exclusion criteria. A search strategy is then developed to guide the systematic search of the literature, covering the following databases: Medline (Ovid), Embase (Ovid), CAB Global Health (Ovid), CINAHL (Ebsco), Cochrane Library, Scopus (Elsevier), and Econlit (Ovid). Sources of grey literature are also searched. Studies that meet the inclusion criteria are systematically coded, extracting data on intervention, outcome, measures, context, geography, equity, and study design. Systematic reviews are also critically appraised using an existing standard checklist. Impact evaluations are not appraised but will be coded according to study design. The process of map-building ends with the creation of an evidence gap map graphic that displays the available evidence according to the intervention and outcome framework of interest. Discussion . Implications arising from the evidence map will be discussed in a separate paper that will summarize findings and make recommendations for the development of a prioritized research agenda.

Newborn health: everybody's business.

Despite advances in issue-attention and in evidence of what works to save newborn lives (e.g., kangaroo mother care, antenatal corticosteroids, immediate and exclusive breastfeeding), we are still falling short on impact. To advance the unfinished newborn survival agenda, newborns must become an integral priority in developing countries where the burden of neonatal mortality is highest. Interventions must be adapted to local contexts and cultures and integrated into packages along the continuum of care delivered through the primary health-care systems that countries have at their disposal.