The RADAR coverage tool: developing a toolkit for rigorous household surveys for reproductive, maternal, newborn, and child health & nutrition indicators.

Population-based intervention coverage data are used to inform the design of projects, programs, and policies and to evaluate their impact. In low- and middle-income countries (LMICs), household surveys are the primary source of coverage data. Many coverage surveys are implemented by organizations with limited experience or resources in population-based data collection. We developed a streamlined survey and set of supporting materials to facilitate rigorous survey design and implementation. The RADAR coverage survey tool aimed to 1) rigorously measure priority reproductive, maternal, newborn, child health & nutrition coverage indicators, and allow for equity and gender analyses; 2) use standard, valid questions, to the extent possible; 3) be as light as possible; 4) be flexible to address users' needs; and 5) be compatible with the Lives Saved Tool for analysis of program impact. Early interactions with stakeholders also highlighted survey planning, implementation, and analysis as challenging areas. We therefore developed a suite of resources to support implementers in these areas. The toolkit was piloted by implementers in Tanzania and in Burkina Faso. Although the toolkit was successfully implemented in these settings and facilitated survey planning and implementation, we found that implementers must still have access to sufficient resources, time, and technical expertise in order to use the tool appropriately. This potentially limits the use of the tool to situations where high-quality surveys or evaluations have been prioritized and adequately resourced.

Creating a Global Legal and Policy Database and Document Repository: Challenges and Lessons Learned From the World Health Organization Sexual, Reproductive, Maternal, Newborn, Child and Adolescent Health Policy Survey.

The World Health Organization (WHO) has collected information on policies on sexual, reproductive, maternal, newborn, child and adolescent health (SRMNCAH) over many years. Creating a global survey that works for every country context is a well-recognized challenge. A comprehensive SRMNCAH policy survey was conducted by WHO from August 2018 through May 2019. WHO regional and country offices coordinated with Ministries of Health and/or national institutions who completed the questionnaire. The survey was completed by 150 of 194 WHO Member States using an online platform that allowed for submission of national source documents. A validation of the responses for selected survey questions against content of the national source documents was conducted for 101 countries (67%) for the first time in the administration of the survey. Data validation draws attention to survey questions that may have been misunderstood or where there was a lot of missing data, but varying methods for validating survey responses against source documents and separate analysis of laws from policies and guidelines may have hindered the overall conclusions of this process. The SRMNCAH policy survey both provided a platform for countries to track their progress in adopting WHO recommendations in national SRMNCAH-related legislation, policies, guidelines and strategies and was used to create a global database and searchable document repository. The outputs of the SRMNCAH policy survey are resources whose importance will be enriched through policy dialogues and wide utilization. Lessons learned from the methodology used for this survey can help to improve future updates and inform similar efforts.

Methods for analysis of complex survey data: an application using the Tanzanian 2015 Demographic and Health Survey and Service Provision Assessment.

BACKGROUND: Low-income and middle-income countries (LMICs) seek to better utilize household and health facility survey data for monitoring and evaluation, as well as for health program planning. However, analysis of this complex survey data are complicated. In Tanzania, the National Evaluation Platform project sought to analyze Demographic and Health Survey (DHS) data and Service Provision Assessment (SPA) data as part of an evaluation of the national One Plan for Maternal and Child Health. To support this evaluation, we used this survey data to answer two key methodological questions: 1) what are the benefits and costs of using sampling weights in rate estimation; and 2) what is the best method for calculating standard errors in these two surveys? METHODS: We conducted a simulation study for each methodologic question. The first simulation study assessed the benefits and costs of using sampling weights in rate estimation. This simulation used weighted and unweighted estimates and examined bias, variance, and the mean squared error (MSE). The second simulation study assessed the best method for calculating standard errors comparing cluster bootstrapped variance estimation, design based asymptotic variance with one level (svy1), and design based asymptotic variance with three levels (svy3). We compared coverage probability and confidence interval length. RESULTS: Our results showed that although weighted estimates were less biased, unweighted estimates were less variable. The weighted estimates had a lower MSE, indicating that the effect of the bias trade-off was greater than the effect of the variance trade-off for most indicators assessed. The best performer for variance estimation was the cluster bootstrap method, followed by the svy3 method. The svy1 method was the worst performer for most indicators assessed. CONCLUSIONS: As complex survey data become more widely used for policymaking in LMICs, there is a need for guidance on the best methods for analyzing this data. The standard of practice has been a design-based analysis using survey weights and the single-level svy method for calculating standard errors. This study puts forth an alternative approach to analysis. In addition, this study offers practical guidance on determining the best method for analysis of complex survey data.

Use of Service Provision Assessments and Service Availability and Readiness Assessments for monitoring quality of maternal and newborn health services in low-income and middle-income countries.

Improving the quality of maternal and newborn health (MNH) services is key to reducing adverse MNH outcomes in low-income and middle-income countries (LMICs). The Service Provision Assessment (SPA) and Service Availability and Readiness Assessment (SARA) are the most widely employed, standardised tools that generate health service delivery data in LMICs. We ascertained the use of SPA/SARA surveys for assessing the quality of MNH services using a two-step approach: a SPA/SARA questionnaire mapping exercise in line with WHO's Quality of Care (QoC) Framework for pregnant women and newborns and the WHO quality standards for care around the time of childbirth; and a scoping literature review, searching for articles that report SPA/SARA data. SPA/SARA surveys are well suited to assess the WHO Framework's cross-cutting dimensions (physical and human resources); SPA also captures elements in the provision and experience of care domains for antenatal care and family planning. Only 4 of 31 proposed WHO quality indicators around the time of childbirth can be fully generated using SPA and SARA surveys, while 19 and 23 quality indicators can be partially obtained from SARA and SPA surveys, respectively; most of these are input indicators. Use of SPA/SARA data is growing, but there is considerable variation in methods employed to measure MNH QoC. With SPA/SARA data available in 30 countries, MNH QoC assessments could benefit from guidance for creating standard metrics. Adding questions in SPA/SARA surveys to assess the WHO QoC Framework's provision and experience of care dimensions would fill significant data gaps in LMICs.

Quality of clinical care and bypassing of primary health centers in India.

In many low and middle-income countries patients often bypass the nearest government health center offering free or subsidized services and seek more expensive care elsewhere. This study examines the role of quality of care, in particular clinician competence and structural quality of the health center, on bypassing behavior. Data for this study comes from a survey of 136 primary health centers (PHCs) and 3517 individuals living in the PHC's immediate vicinity in rural Chhattisgarh, India. Overall, the majority (67%) of patients bypassed the local PHC when seeking treatment. Bypassing decreased as provider competence increased, up to a point, after which, improvements in competency did not reduce bypassing. The clinical competence of the health care provider had a greater effect on reducing bypassing compared to PHC structural quality such as the building condition and drug stock-outs. However, the regular presence of clinical providers in the PHC was associated with lower bypassing. Patients that visited the local PHC spent half as much out-of-pocket as those that were treated at private clinics. Poor patients were less likely to bypass the local PHC compared to non-poor patients. These findings suggest that improving structural quality is not sufficient to reduce bypassing of PHCs. While better provider competency can substantially reduce bypassing, beyond a threshold competency level there is little effect. Efforts to strengthen facility-based primary care services need to go beyond simply focusing on improving infrastructure or quality of clinical care. There is a need to rethink how PHCs can be made more relevant to the health care needs of the communities they serve.

Monitoring service delivery for universal health coverage: the Service Availability and Readiness Assessment.

OBJECTIVE: To describe the Service Availability and Readiness Assessment (SARA) and the results of its implementation in six countries across three continents. METHODS: The SARA is a comprehensive approach for assessing and monitoring health service availability and the readiness of facilities to deliver health-care interventions, with a standardized set of indicators that cover all main programmes. Standardized data-collection instruments are used to gather information on a defined set of selected tracer items from public and private health facilities through a facility sample survey or census. Results from assessments in six countries are shown. FINDINGS: The results highlight important gaps in service delivery that are obstacles to universal access to health services. Considerable variation was found within and across countries in the distribution of health facility infrastructure and workforce and in the types of services offered. Weaknesses in laboratory diagnostic capacities and gaps in essential medicines and commodities were common across all countries. CONCLUSION: The SARA fills an important information gap in monitoring health system performance and universal health coverage by providing objective and regular information on all major health programmes that feeds into country planning cycles.