Measurement for Change: Reflections from innovators' experiences with monitoring, evaluation, and learning systems for Early Childhood Development.

In this review paper, we explore how on-the-ground Early Childhood Development (ECD) innovators are using monitoring, evaluation, and learning (MEL) systems to guide the design and implementation of ECD programs, as well as how MEL systems can influence policy and support the achievement of impact at scale. We reflect on articles in the Frontiers series "Effective delivery of integrated interventions in early childhood: innovations in evidence use, monitoring, evaluation, and learning." The 31 contributions to the series reflect the breadth and depth of complexity that characterizes ECD, including global geographic spread, with studies from Asia, Europe, Africa, and Latin America and the Caribbean. Our synthesis finds that integrating MEL processes and systems into the fabric of a program or policy initiative can broaden the underlying value proposition. Specifically, ECD organizations sought to design their MEL systems to ensure programs fit the values, goals, experiences and conceptual frameworks of diverse stakeholders, so that participating makes sense to all. For example, formative, exploratory research identified the priorities and needs of the target population and frontline service providers, and informed the content and delivery of an intervention. ECD organizations also designed their MEL systems to support a shift of accountability toward broader ownership: They included delivery agents and program participants alike as subjects rather than objects, through active participation in data collection, and by providing opportunities for equitable discussion of results and decision-making. Programs collected data to respond to specialized characteristics, priorities and needs, embedding program activities into existing day-to-day routines. Further, papers pointed to the importance of intentionally involving a variety of stakeholders in national and international dialogues to ensure that diverse ECD data collection efforts are aligned and multiple perspectives are considered in the development of national ECD policies. And, several papers illustrate the value of creative methods and measurement tools to integrate MEL into a program or policy initiative. Finally, our synthesis concludes that these findings align with the five aspirations that were formulated as part of the Measurement for Change dialogue, which motivated the launch of the series.

Leveraging monitoring, evaluation, and learning to scale the Enabling Inclusion® program for children with disabilities in India and globally.

Introduction: Children with disabilities in low- and middle-income countries face many challenges and lack adequate services, including access to rehabilitation professionals. To address this lack of access, Amar Seva Sangam Ayikudy (ASSA), a non-governmental organization (NGO) in India, designed a technology-leveraged rehabilitation program called Enabling Inclusion® (EI®), and implemented it in one state (Tamil Nadu, India) before scaling it. The model is supported by the EI® app, which enables organizations to screen, assess and monitor progress of children with disabilities via rehabilitation specialists and community rehabilitation workers, and to provide family-centered, goal-based interventions. An extensive monitoring, evaluation, and learning (MEL) framework is embedded into the program. This paper explores how this MEL system supported the scaling of the EI® model, reaching additional beneficiaries nationally and globally. Methods: This paper describes ASSA's MEL framework and demonstrates its use for decision-making in the process of scaling. It also explores how collaborations with various government departments, NGOs, and private partners contributed to the scaling of the EI® model and technology. Results: Scaling of the EI® program was achieved by (1) expansion of the program in rural Tamil Nadu (vertical scale-up) in partnership with the Tamil Nadu government and private partners, and (2) by licensing the EI® app and model to other NGOs in various states in India and globally (horizontal scale-up). Systematic examination of key program and performance indicators, as well as stakeholder feedback, informed decisions to modify the EI® app over time. This included further customizing to the needs of children and service providers, covering a greater range of age groups and contexts, and modifying service delivery models. Child functional independence, participation, and inclusion was further strengthened by mobilizing parent empowerment groups, community awareness programs, school advocacy, and entitlements from the government. Flexibility in the implementation model of the EI® app allowed for adaptation to local contexts and organizations, and facilitated its scale-up. Conclusion: A dynamic, inclusive, and locally grounded MEL system, a flexible and collaborative approach, and an adaptive implementation model increased the accessibility of an early intervention and childhood rehabilitation program for children with disabilities and their families throughout the state of Tamil Nadu, across India, and internationally.

Assessing the Impact of an Intervention Project by the Young women's Christian Association of Malawi on Psychosocial Well-Being of Adolescent Mothers and Their Children in Malawi.

Adolescent mothers in Malawi face psychosocial challenges such as low resilience level, low self-esteem, poor maternal-infant interaction, and exposure to intimate partner violence (IPV). Children of adolescent mothers often face numerous risks such as low birth-weight, stunted growth, infant death, low school enrolment, increased grade repetition, and dropouts that put them at greater risk of poor developmental outcomes and socio-emotional problems. This study assessed the impact of components of a community project conducted by the Young Women's Christian association of Malawi in providing psychosocial support to adolescent mothers and their children. The goals of the project were; (1) to improve early childhood development in babies born to adolescent mothers; and (2) to enhance the psychosocial well-being of adolescent mothers (self-esteem, resilience stress, and parenting skills). This descriptive mixed methods evaluation study comprised an intervention and control groups of adolescent mothers respectively. The project had 3 centers in southern region districts of Malawi. Target population was adolescent mothers 18 years of age and below. At baseline we enrolled 267 mothers and at the end of the project we had 211 mothers. The project involved monthly meetings with adolescent mothers imparting knowledge and skills and early childhood education activities. From July 2017 to June 2019, 58 sessions were conducted. In the first year the control group had no meetings, however they received the intervention in the second year. Overall results in the intervention group showed statistically significant increase in knowledge on parenting skills (p < 0.01), nutritional practice (p < 0.01), motor skills and cognitive functions in children (p < 0.01) as well as expressive language and socio-emotional capacities in children (p < 0.01), while the change in confidence and psychosocial well-being was not statistically significant (p = 0.8823). Community projects such as these enhance parenting skills and improve development of children born to adolescent mothers. Improving psychosocial support is complex and requires further research and a more holistic approach.

Using Evidence and Data to Design an Intervention in the Project Community Model for Fostering Health and Wellbeing Among Adolescent Mothers and Their Children.

In this paper, quantitative and qualitative measurements of maternal psychosocial wellbeing were utilized in three districts in Malawi that guided decision-making to increase the wellbeing of adolescent mothers and promote the healthy upbringing of their children. The 1-year design stage of the study relied on several sources of information: literature search, prior project implementation of similar projects, discussions with officials at the Malawi Department of Social Welfare, and observation visits in the targeted districts. The approaches for collecting data mentioned were triangulated for the development of a baseline survey. The baseline survey generated systematically collected data of the experiences and recalls as well as the missing data from the preliminary evaluation of the existing data. The baseline data gave the Young Women's Christian Association (YWCA) insight on the type of intervention required in order to give a greater and more holistic effect on the beneficiaries. We also discuss the lessons we learned as to whether the assumptions we had made at the onset were correct. If they were not correct, we explained the measures we took to correct the design or implementation of the project. Finally, the data provided benchmarks for project monitoring and evaluation.

Rapid-Cycle Evaluation in an Early Intervention Program for Children With Developmental Disabilities in South India: Optimizing Service Providers' Quality of Work-Life, Family Program Engagement, and School Enrollment.

Background: This paper explores how implementation and refinement of an early intervention (EI) program for children with delayed development was informed by an iterative, intentional and structured process of measurement. Providing access to early intervention therapy for children in rural areas of India is challenging due to a lack of rehabilitation therapists and programs. Following a biopsychosocial framework and principles of community-based rehabilitation, a non-governmental organization, Amar Seva Sangam (ASSA), overcame those barriers by designing a digital technology supported EI program in rural Tamil Nadu, India. Program objectives included providing service access; supporting program engagement, child development and school enrollment; and positioning the intervention for scale-up. This paper contributes to a growing body of literature on how program design and implementation can be informed through a cyclical process of data collection, analysis, reflection, and adaptation. Methods: Through several strands of data collection, the design and implementation of the EI program was adapted and improved. This included qualitative data from focus groups and interviews with caregivers and service providers, and a mobile application that collected and monitored longitudinal quantitative data, including program engagement rates, developmental progression, caregiver outcomes, and school enrollment status. Results: Measurements throughout the program informed decision-making by identifying facilitators and barriers to service providers' quality of work-life, family program engagement, and school enrollment. Consultation with key stakeholders, including caregivers and service providers, and data driven decision making led to continual program changes that improved service provider quality of work-life, program engagement and school enrollment. These changes included addressing gender-related work challenges for service providers; forming caregiver support networks; introducing psychological counseling for caregivers; providing medical consultations and assistive devices; creating community awareness programs; improving access to therapy services; focusing on caregiver education, motivation and support; and advocacy for accessibility in schools. Conclusion: The process of using evidence-informed and stakeholder driven adaptations to the early intervention program, led to improved service provider quality of work-life, greater program engagement, improved school enrollment and positioned the intervention for scale-up, providing lessons that may be beneficial in other contexts.

The Aspirations of Measurement for Change.

This Perspective presents the five key aspirations of an approach to data use, decision making and monitoring, evaluation, and learning (MEL) in Early Childhood Development (ECD) referred to as Measurement for Change. The core ideas of Measurement for Change gave rise to this series, and many of the papers submitted in this series speak to this approach, whether directly or indirectly. The five aspirations describe interconnected concepts that advocate for practitioners and researchers within ECD to build the capacity to use data in their decision making, by establishing a monitoring, evaluation, and learning system that strives to be: Dynamic; Inclusive; Informative; Interactive; and People-centered.

Measurement for Change: From Idea to Approach.

Measurement for Change proposes an integration of monitoring, evaluation, and learning into decision-making systems that support sustainable transition of interventions to scale. It was developed using a cyclical, interactive 1-year dialogue between early childhood development (ECD) practitioners and academics from across the globe. Details are presented in Krapels et al. (1) as part of this special issue in Frontiers. In this paper, we trace the developments that inspired Measurement for Change and the novel ways in which the approach and the special issue was developed. The experience, and the reflections on this experience, are intended to inform those implementing initiatives that similarly seek to integrate practitioner- and academic experiences in support of sustainable transitions of interventions to scale.