Measurement for Change: Reflections from innovators' experiences with monitoring, evaluation, and learning systems for Early Childhood Development.

In this review paper, we explore how on-the-ground Early Childhood Development (ECD) innovators are using monitoring, evaluation, and learning (MEL) systems to guide the design and implementation of ECD programs, as well as how MEL systems can influence policy and support the achievement of impact at scale. We reflect on articles in the Frontiers series "Effective delivery of integrated interventions in early childhood: innovations in evidence use, monitoring, evaluation, and learning." The 31 contributions to the series reflect the breadth and depth of complexity that characterizes ECD, including global geographic spread, with studies from Asia, Europe, Africa, and Latin America and the Caribbean. Our synthesis finds that integrating MEL processes and systems into the fabric of a program or policy initiative can broaden the underlying value proposition. Specifically, ECD organizations sought to design their MEL systems to ensure programs fit the values, goals, experiences and conceptual frameworks of diverse stakeholders, so that participating makes sense to all. For example, formative, exploratory research identified the priorities and needs of the target population and frontline service providers, and informed the content and delivery of an intervention. ECD organizations also designed their MEL systems to support a shift of accountability toward broader ownership: They included delivery agents and program participants alike as subjects rather than objects, through active participation in data collection, and by providing opportunities for equitable discussion of results and decision-making. Programs collected data to respond to specialized characteristics, priorities and needs, embedding program activities into existing day-to-day routines. Further, papers pointed to the importance of intentionally involving a variety of stakeholders in national and international dialogues to ensure that diverse ECD data collection efforts are aligned and multiple perspectives are considered in the development of national ECD policies. And, several papers illustrate the value of creative methods and measurement tools to integrate MEL into a program or policy initiative. Finally, our synthesis concludes that these findings align with the five aspirations that were formulated as part of the Measurement for Change dialogue, which motivated the launch of the series.

The Aspirations of Measurement for Change.

This Perspective presents the five key aspirations of an approach to data use, decision making and monitoring, evaluation, and learning (MEL) in Early Childhood Development (ECD) referred to as Measurement for Change. The core ideas of Measurement for Change gave rise to this series, and many of the papers submitted in this series speak to this approach, whether directly or indirectly. The five aspirations describe interconnected concepts that advocate for practitioners and researchers within ECD to build the capacity to use data in their decision making, by establishing a monitoring, evaluation, and learning system that strives to be: Dynamic; Inclusive; Informative; Interactive; and People-centered.

Human resources and curricula content for early child development implementation: multicountry mixed methods evaluation.

OBJECTIVE: The WHO recommends responsive caregiving and early learning (RCEL) interventions to improve early child development (ECD), and to achieve the Sustainable Development Goals' vision of a world where all children thrive. Implementation of RCEL programmes in low and middle-income countries (LMIC) requires evidence to inform decisions about human resources and curricula content. We aimed to describe human resources and curricula content for implementation of RCEL projects across diverse LMICs, using data from the Grand Challenges Canada Saving Brains ECD portfolio. SETTING: We evaluated 32 RCEL projects across 17 LMICs on four continents. PARTICIPANTS: Overall, 2165 workers delivered ECD interventions to 25 909 families. INTERVENTION: Projects were either stand-alone RCEL or RCEL combined with health and nutrition, and/or safety and security. PRIMARY AND SECONDARY OUTCOMES: We undertook a mixed methods evaluation of RCEL projects within the Saving Brains portfolio. Quantitative data were collected through standardised reporting tools. Qualitative data were collected from ECD experts and stakeholders and analysed using thematic content analysis, informed by literature review. RESULTS: Major themes regarding human resources included: worker characteristics, incentivisation, retention, training and supervision, and regarding curricula content: flexible adaptation of content and delivery, fidelity, and intervention duration and dosage. Lack of an agreed standard ECD package contributed to project heterogeneity. Incorporation of ECD into existing services may facilitate scale-up but overburdened workers plus potential reductions in service quality remain challenging. Supportive training and supervision, inducement, worker retention, dosage and delivery modality emerged as key implementation decisions. CONCLUSIONS: This mixed methods evaluation of a multicountry ECD portfolio identified themes for consideration by policymakers and programme leaders relevant to RCEL implementation in diverse LMICs. Larger studies, which also examine impact, including high-quality process and costing evaluations with comparable data, are required to further inform decisions for implementation of RCEL projects at national and regional scales.

Contextual design choices and partnerships for scaling early child development programmes.

Translating the Nurturing Care Framework and unprecedented global policy support for early child development (ECD) into action requires evidence-informed guidance about how to implement ECD programmes at national and regional scale. We completed a literature review and participatory mixed-method evaluation of projects in Saving Brains®, Grand Challenges Canada® funded ECD portfolio across 23 low- and middle-income countries (LMIC). Using an adapted programme cycle, findings from evaluation related to partnerships and leadership, situational analyses, and design for scaling ECD were considered. 39 projects (5 'Transition to Scale' and 34 'Seed') were evaluated. 63% were delivered through health and 84% focused on Responsive Caregiving and Early Learning (RCEL). Multilevel partnerships, leadership and targeted situational analysis were crucial to design and adaptation. A theory of change approach to consider pathways to impact was useful for design, but practical situational analysis tools and local data to guide these processes were lacking. Several RCEL programmes, implemented within government services, had positive impacts on ECD outcomes and created more enabling caregiving environments. Engagement of informal and private sectors provided an alternative approach for reaching children where government services were sparse. Cost-effectiveness was infrequently measured. At small-scale RCEL interventions can be successfully adapted and implemented across diverse settings through processes which are responsive to situational analysis within a partnership model. Accelerating progress will require longitudinal evaluation of ECD interventions at much larger scale, including programmes targeting children with disabilities and humanitarian settings with further exploration of cost-effectiveness, critical content and human resources.

Counting outcomes, coverage and quality for early child development programmes.

Improved measurement in early child development (ECD) is a strategic focus of the WHO, UNICEF and World Bank Nurturing Care Framework. However, evidence-based approaches to monitoring and evaluation (M&E) of ECD projects in low-income and middle-income countries (LMIC) are lacking. The Grand Challenges Canada®-funded Saving Brains® ECD portfolio provides a unique opportunity to explore approaches to M&E of ECD programmes across diverse settings. Focused literature review and participatory mixed-method evaluation of the Saving Brains portfolio was undertaken using an adapted impact framework. Findings related to measurement of quality, coverage and outcomes for scaling ECD were considered. Thirty-nine ECD projects implemented in 23 LMIC were evaluated. Projects used a 'theory of change' based M&E approach to measure a range of inputs, outputs and outcomes. Over 29 projects measured cognitive, language, motor and socioemotional outcomes. 18 projects used developmental screening tools to measure outcomes, with a trade-off between feasibility and preferred practice. Environmental inputs such as the home environment were measured in 15 projects. Qualitative data reflected the importance of measurement of project quality and coverage, despite challenges measuring these constructs across contexts. Improved measurement of intervention quality and measurement of coverage, which requires definition of the numerator (ie, intervention) and denominator (ie, population in need/at risk), are needed for scaling ECD programmes. Innovation in outcome measurement, including intermediary outcome measures that are feasible and practical to measure in routine services, is also required, with disaggregation to better target interventions to those most in need and ensure that no child is left behind.

Scaling early child development: what are the barriers and enablers?

The Sustainable Development Goals, Global Strategy for Women's, Children's and Adolescents' Health (2016-2030) and Nurturing Care Framework all include targets to ensure children thrive However, many projects to support early childhood development (ECD) do not 'scale well' and leave large numbers of children unreached. This paper is the fifth in a series examining effective scaling of ECD programmes. This qualitative study explored experiences of scaling-up among purposively recruited implementers of ECD projects in low- and middle-income countries. Participants were sampled, by means of snowball sampling, from existing networks notably through Saving Brains®, Grand Challenges Canada®. Findings of a recent literature review on scaling-up frameworks, by the WHO, informed the development of a semistructured interview schedule. All interviews were conducted in English, via Skype, audio recorded and transcribed verbatim. Interviews were analysed using framework analysis. Framework analysis identified six major themes based on a standard programme cycle: planning and strategic choices, project design, human resources, financing and resource mobilisation, monitoring and evaluation, and leadership and partnerships. Key informants also identified an overarching theme regarding what scaling-up means. Stakeholders have not found existing literature and available frameworks helpful in guiding them to successful scale-up. Our research suggests that rather than proposing yet more theoretical guidelines or frameworks, it would be better to support stakeholders in developing organisational leadership capacity and partnership strategies to enable them to effectively apply a practical programme cycle or systematic process in their own contexts.

Practical considerations for transitioning early childhood interventions to scale: lessons from the Saving Brains portfolio.

Small pilot studies of young children have frequently shown promise, but very few have been successfully scaled to the regional or national levels. How can we ensure that these promising approaches move from a suite of pilots to full-scale implementation that can deliver sustainable impact for hundreds of millions of children? To elucidate concrete lessons learned and suggestions on accelerating the transition to impact at scale, we reviewed the Saving Brains portfolio to better understand three points: (1) the extent to which useful signals of impact could be extracted from data at the seed phase, (2) the ways in which innovators (project leaders) were approaching human resource challenges critical for scaling, and (3) the multisector diversity of the portfolio and the way innovators entered partnerships. The findings suggest key considerations for transitioning early childhood development interventions to scale and sustainability: strong entrepreneurial leadership, rigorous measurement and active use of data in support of adaptive learning, and champions acting at subnational levels. Together, these can enable flexible, iterative learning that can make the scaling process an opportunity to increase the level of benefit each child receives from an intervention.