Estimating the health impact of vaccination against ten pathogens in 98 low-income and middle-income countries from 2000 to 2030: a modelling study.

BACKGROUND: The past two decades have seen expansion of childhood vaccination programmes in low-income and middle-income countries (LMICs). We quantify the health impact of these programmes by estimating the deaths and disability-adjusted life-years (DALYs) averted by vaccination against ten pathogens in 98 LMICs between 2000 and 2030. METHODS: 16 independent research groups provided model-based disease burden estimates under a range of vaccination coverage scenarios for ten pathogens: hepatitis B virus, Haemophilus influenzae type B, human papillomavirus, Japanese encephalitis, measles, Neisseria meningitidis serogroup A, Streptococcus pneumoniae, rotavirus, rubella, and yellow fever. Using standardised demographic data and vaccine coverage, the impact of vaccination programmes was determined by comparing model estimates from a no-vaccination counterfactual scenario with those from a reported and projected vaccination scenario. We present deaths and DALYs averted between 2000 and 2030 by calendar year and by annual birth cohort. FINDINGS: We estimate that vaccination of the ten selected pathogens will have averted 69 million (95% credible interval 52-88) deaths between 2000 and 2030, of which 37 million (30-48) were averted between 2000 and 2019. From 2000 to 2019, this represents a 45% (36-58) reduction in deaths compared with the counterfactual scenario of no vaccination. Most of this impact is concentrated in a reduction in mortality among children younger than 5 years (57% reduction [52-66]), most notably from measles. Over the lifetime of birth cohorts born between 2000 and 2030, we predict that 120 million (93-150) deaths will be averted by vaccination, of which 58 million (39-76) are due to measles vaccination and 38 million (25-52) are due to hepatitis B vaccination. We estimate that increases in vaccine coverage and introductions of additional vaccines will result in a 72% (59-81) reduction in lifetime mortality in the 2019 birth cohort. INTERPRETATION: Increases in vaccine coverage and the introduction of new vaccines into LMICs have had a major impact in reducing mortality. These public health gains are predicted to increase in coming decades if progress in increasing coverage is sustained. FUNDING: Gavi, the Vaccine Alliance and the Bill & Melinda Gates Foundation.

Validation of MINORMIX Approach for Estimation of Low Birthweight Prevalence Using a Rural Nepal Dataset.

BACKGROUND: The Global Nutrition Target of reducing low birthweight (LBW) by ≥30% between 2012 and 2025 has led to renewed interest in producing accurate, population-based, national LBW estimates. Low- and middle-income countries rely on household surveys for birthweight data. These data are frequently incomplete and exhibit strong "heaping." Standard survey adjustment methods produce estimates with residual bias. The global database used to report against the LBW Global Nutrition Target adjusts survey data using a new MINORMIX (multiple imputation followed by normal mixture) approach: 1) multiple imputation to address missing birthweights, followed by 2) use of a 2-component normal mixture model to account for heaping of birthweights. OBJECTIVES: To evaluate the performance of the MINORMIX birthweight adjustment approach and alternative methods against gold-standard measured birthweights in rural Nepal. METHODS: As part of a community-randomized trial in rural Nepal, we measured "gold-standard" birthweights at birth and returned 1-24 mo later to collect maternally reported birthweights using standard survey methods. We compared estimates of LBW from maternally reported data derived using: 1) the new MINORMAX approach; 2) the previously used Blanc-Wardlaw adjustment; or 3) no adjustment for missingness or heaping against our gold standard. We also assessed the independent contribution of multiple imputation and curve fitting to LBW adjustment. RESULTS: Our gold standard found 27.7% of newborns were LBW. The unadjusted LBW estimate based on maternal report with simulated missing birthweights was 14.5% (95% CI: 11.6, 18.0%). Application of the Blanc-Wardlaw adjustment increased the LBW estimate to 20.6%. The MINORMIX approach produced an estimate of 26.4% (95% CI: 23.5, 29.3%) LBW, closest to and with bounds encompassing the measured point estimate. CONCLUSIONS: In a rural Nepal validation dataset, the MINORMIX method generated a more accurate LBW estimate than the previously applied adjustment method. This supports the use of the MINORMIX method to produce estimates for tracking the LBW Global Nutrition Target.

The effect of sampling health facilities on estimates of effective coverage: a simulation study.

BACKGROUND: Most existing facility assessments collect data on a sample of health facilities. Sampling of health facilities may introduce bias into estimates of effective coverage generated by ecologically linking individuals to health providers based on geographic proximity or administrative catchment. METHODS: We assessed the bias introduced to effective coverage estimates produced through two ecological linking approaches (administrative unit and Euclidean distance) applied to a sample of health facilities. Our analysis linked MICS household survey data on care-seeking for child illness and childbirth care with data on service quality collected from a census of health facilities in the Savanes region of Cote d'Ivoire. To assess the bias introduced by sampling, we drew 20 random samples of three different sample sizes from our census of health facilities. We calculated effective coverage of sick child and childbirth care using both ecological linking methods applied to each sampled facility data set. We compared the sampled effective coverage estimates to ecologically linked census-based estimates and estimates based on true source of care. We performed sensitivity analyses with simulated preferential care-seeking from higher-quality providers and randomly generated provider quality scores. RESULTS: Sampling of health facilities did not significantly bias effective coverage compared to either the ecologically linked estimates derived from a census of facilities or true effective coverage estimates using the original data or simulated random quality sensitivity analysis. However, a few estimates based on sampling in a setting where individuals preferentially sought care from higher-quality providers fell outside of the estimate bounds of true effective coverage. Those cases predominantly occurred using smaller sample sizes and the Euclidean distance linking method. None of the sample-based estimates fell outside the bounds of the ecologically linked census-derived estimates. CONCLUSIONS: Our analyses suggest that current health facility sampling approaches do not significantly bias estimates of effective coverage produced through ecological linking. Choice of ecological linking methods is a greater source of bias from true effective coverage estimates, although facility sampling can exacerbate this bias in certain scenarios. Careful selection of ecological linking methods is essential to minimize the potential effect of both ecological linking and sampling error.

Reliability of maternal recall of delivery and immediate newborn care indicators in Sarlahi, Nepal.

BACKGROUND: The intrapartum period is a time of high mortality risk for newborns and mothers. Numerous interventions exist to minimize risk during this period. Data on intervention coverage are needed for health system improvement. Maternal report of intrapartum interventions through surveys is the primary source of coverage data, but they may be invalid or unreliable. METHODS: We assessed the reliability of maternal report of delivery and immediate newborn care for a sample of home and health facility births in Sarlahi, Nepal. Mothers were visited as soon as possible following delivery (< 72 h) and asked to report circumstances of labor and delivery. A subset was revisited 1-24 months after delivery and asked to recall interventions received using standard household survey questions. We assessed the reliability of each indicator by comparing what mothers reported immediately after delivery against what they reported at the follow-up survey. We assessed potential variation in reliability of maternal report by characteristics of the mother, birth event, or intervention prevalence. RESULTS: One thousand five hundred two mother/child pairs were included in the reliability study, with approximately half of births occurring at home. A higher proportion of women who delivered in facilities reported "don't know" when asked to recall specific interventions both initially and at follow-up. Most indicators had high observed percent agreement, but kappa values were below 0.4, indicating agreement was primarily due to chance. Only "received any injection during delivery" demonstrated high reliability among all births (kappa: 0.737). The reliability of maternal report was typically lower among women who delivered at a facility. There was no difference in reliability based on time since birth of the follow-up interview. We observed over-reporting of interventions at follow-up that were more common in the population and under-reporting of less common interventions. CONCLUSIONS: This study reinforces previous findings that mothers are unable to report reliably on many interventions within the peripartum period. Household surveys which rely on maternal report, therefore, may not be an appropriate method for collecting data on coverage of many interventions during the peripartum period. This is particularly true among facility births, where many interventions may occur without the mother's full knowledge.

Impact of imprecise household location on effective coverage estimates generated through linking household and health provider data by geographic proximity: a simulation study.

BACKGROUND: Geographic proximity is often used to link household and health provider data to estimate effective coverage of health interventions. Existing household surveys often provide displaced data on the central point within household clusters rather than household location. This may introduce error into analyses based on the distance between households and providers. METHODS: We assessed the effect of imprecise household location on quality-adjusted effective coverage of child curative services estimated by linking sick children to providers based on geographic proximity. We used data on care-seeking for child illness and health provider quality in Southern Province, Zambia. The dataset included the location of respondent households, a census of providers, and data on the exact outlets utilized by sick children included in the study. We displaced the central point of each household cluster point five times. We calculated quality-adjusted coverage by assigning each sick child to a provider's care based on three measures of geographic proximity (Euclidean distance, travel time, and geographic radius) from the household location, cluster point, and displaced cluster locations. We compared the estimates of quality-adjusted coverage to each other and estimates using each sick child's true source of care. We performed sensitivity analyses with simulated preferential care-seeking from higher-quality providers and randomly generated provider quality scores. RESULTS: Fewer children were linked to their true source of care using cluster locations than household locations. Effective coverage estimates produced using undisplaced or displaced cluster points did not vary significantly from estimates produced using household location data or each sick child's true source of care. However, the sensitivity analyses simulating greater variability in provider quality showed bias in effective coverage estimates produced with the geographic radius and travel time method using imprecise location data in some scenarios. CONCLUSIONS: Use of undisplaced or displaced cluster location reduced the proportion of children that linked to their true source of care. In settings with minimal variability in quality within provider categories, the impact on effective coverage estimates is limited. However, use of imprecise household location and choice of geographic linking method can bias estimates in areas with high variability in provider quality or preferential care-seeking.

Estimating c-section coverage: Assessing method performance and characterizing variations in coverage.

Background: Cesarean section (c-section) is an essential tool for preventing, stillbirths, maternal, and newborn death. However, data on coverage of medically necessary c-section is limited in low- and middle-income settings. Methods: We estimated national c-section coverage using household survey data from 98 low- and middle-income countries. To disaggregate elective and medically necessary c-sections, we estimated the proportion of women in each survey wealth quintile who gave birth via c-section assuming a denominator that 12.5% of births necessitate a c-section delivery. We capped stratum coverage at 100%. We estimated national c-section coverage weighting for the proportion of births occurring in each wealth quintile. We examined 1) variation in estimated c-section by wealth quintile, national income classification, and stage in the obstetric transition, 2) how varying definitions impact the classification of countries' access to c-section, and 3) correlation between c-section and related mortality outcomes. Results: Both increasing national and household wealth are associated with increasing levels of c-section coverage and c-section rate. C-section coverage was highly inequitable by wealth within a country. Differentials in coverage were most pronounced in countries with c-section rates below 10%; however, some countries showed significant gaps in c-section coverage in poor subpopulations despite high c-section rates nationally. The choice of indicator and threshold altered whether a country was classified as having adequate access to c-section services. C-section coverage estimates showed a stronger relationship with closely related health outcomes than national c-section rates. Conclusions: Generating estimates of c-section coverage is crucial for gauging gaps in c-section access. Our approach for calculating c-section coverage using stratification by wealth to adjust for potential elective c-sections is supported by the strong correlations between household wealth and subnational c-section rate, and the association between our coverage estimates and health outcomes at a national level. Looking at national c-section rates alone may paint an inaccurate picture of c-section access and mask subnational inequities in coverage. The need to accurately measure access to c-section will continue to increase as growth in LMICs drives inequities in coverage and introduces dual concerns related to c-section overuse in some populations while others lack access to care.

Impact of the Early Stages of the COVID-19 Pandemic on Coverage of Reproductive, Maternal, and Newborn Health Interventions in Ethiopia: A Natural Experiment.

Background: The COVID-19 pandemic and response have the potential to disrupt access and use of reproductive, maternal, and newborn health (RMNH) services. Numerous initiatives aim to gauge the indirect impact of COVID-19 on RMNH. Methods: We assessed the impact of COVID-19 on RMNH coverage in the early stages of the pandemic using panel survey data from PMA-Ethiopia. Enrolled pregnant women were surveyed 6-weeks post-birth. We compared the odds of service receipt, coverage of RMNCH service indicators, and health outcomes within the cohort of women who gave birth prior to the pandemic and the COVID-19 affected cohort. We calculated impacts nationally and by urbanicity. Results: This dataset shows little disruption of RMNH services in Ethiopia in the initial months of the pandemic. There were no significant reductions in women seeking health services or the content of services they received for either preventative or curative interventions. In rural areas, a greater proportion of women in the COVID-19 affected cohort sought care for peripartum complications, ANC, PNC, and care for sick newborns. Significant reductions in coverage of BCG vaccination and chlorohexidine use in urban areas were observed in the COVID-19 affected cohort. An increased proportion of women in Addis Ababa reported postpartum family planning in the COVID-19 affected cohort. Despite the lack of evidence of reduced health services, the data suggest increased stillbirths in the COVID-19 affected cohort. Discussion: The government of Ethiopia's response to control the COVID-19 pandemic and ensure continuity of essential health services appears to have successfully averted most negative impacts on maternal and neonatal care. This analysis cannot address the later effects of the pandemic and may not capture more acute or geographically isolated reductions in coverage. Continued efforts are needed to ensure that essential health services are maintained and even strengthened to prevent indirect loss of life.

Methodological considerations for linking household and healthcare provider data for estimating effective coverage: a systematic review.

OBJECTIVE: To assess existing knowledge related to methodological considerations for linking population-based surveys and health facility data to generate effective coverage estimates. Effective coverage estimates the proportion of individuals in need of an intervention who receive it with sufficient quality to achieve health benefit. DESIGN: Systematic review of available literature. DATA SOURCES: Medline, Carolina Population Health Center and Demographic and Health Survey publications and handsearch of related or referenced works of all articles included in full text review. The search included publications from 1 January 2000 to 29 March 2021. ELIGIBILITY CRITERIA: Publications explicitly evaluating (1) the suitability of data, (2) the implications of the design of existing data sources and (3) the impact of choice of method for combining datasets to obtain linked coverage estimates. RESULTS: Of 3805 papers reviewed, 70 publications addressed relevant issues. Limited data suggest household surveys can be used to identify sources of care, but their validity in estimating intervention need was variable. Methods for collecting provider data and constructing quality indices were diverse and presented limitations. There was little empirical data supporting an association between structural, process and outcome quality. Few studies addressed the influence of the design of common data sources on linking analyses, including imprecise household geographical information system data, provider sampling design and estimate stability. The most consistent evidence suggested under certain conditions, combining data based on geographical proximity or administrative catchment (ecological linking) produced similar estimates to linking based on the specific provider utilised (exact match linking). CONCLUSIONS: Linking household and healthcare provider data can leverage existing data sources to generate more informative estimates of intervention coverage and care. However, existing evidence on methods for linking data for effective coverage estimation are variable and numerous methodological questions remain. There is need for additional research to develop evidence-based, standardised best practices for these analyses.

Lives saved with vaccination for 10 pathogens across 112 countries in a pre-COVID-19 world.

Background: Vaccination is one of the most effective public health interventions. We investigate the impact of vaccination activities for Haemophilus influenzae type b, hepatitis B, human papillomavirus, Japanese encephalitis, measles, Neisseria meningitidis serogroup A, rotavirus, rubella, Streptococcus pneumoniae, and yellow fever over the years 2000-2030 across 112 countries. Methods: Twenty-one mathematical models estimated disease burden using standardised demographic and immunisation data. Impact was attributed to the year of vaccination through vaccine-activity-stratified impact ratios. Results: We estimate 97 (95%CrI[80, 120]) million deaths would be averted due to vaccination activities over 2000-2030, with 50 (95%CrI[41, 62]) million deaths averted by activities between 2000 and 2019. For children under-5 born between 2000 and 2030, we estimate 52 (95%CrI[41, 69]) million more deaths would occur over their lifetimes without vaccination against these diseases. Conclusions: This study represents the largest assessment of vaccine impact before COVID-19-related disruptions and provides motivation for sustaining and improving global vaccination coverage in the future. Funding: VIMC is jointly funded by Gavi, the Vaccine Alliance, and the Bill and Melinda Gates Foundation (BMGF) (BMGF grant number: OPP1157270 / INV-009125). Funding from Gavi is channelled via VIMC to the Consortium's modelling groups (VIMC-funded institutions represented in this paper: Imperial College London, London School of Hygiene and Tropical Medicine, Oxford University Clinical Research Unit, Public Health England, Johns Hopkins University, The Pennsylvania State University, Center for Disease Analysis Foundation, Kaiser Permanente Washington, University of Cambridge, University of Notre Dame, Harvard University, Conservatoire National des Arts et Métiers, Emory University, National University of Singapore). Funding from BMGF was used for salaries of the Consortium secretariat (authors represented here: TBH, MJ, XL, SE-L, JT, KW, NMF, KAMG); and channelled via VIMC for travel and subsistence costs of all Consortium members (all authors). We also acknowledge funding from the UK Medical Research Council and Department for International Development, which supported aspects of VIMC's work (MRC grant number: MR/R015600/1).JHH acknowledges funding from National Science Foundation Graduate Research Fellowship; Richard and Peggy Notebaert Premier Fellowship from the University of Notre Dame. BAL acknowledges funding from NIH/NIGMS (grant number R01 GM124280) and NIH/NIAID (grant number R01 AI112970). The Lives Saved Tool (LiST) receives funding support from the Bill and Melinda Gates Foundation.This paper was compiled by all coauthors, including two coauthors from Gavi. Other funders had no role in study design, data collection, data analysis, data interpretation, or writing of the report. All authors had full access to all the data in the study and had final responsibility for the decision to submit for publication.

Early estimates of the indirect effects of the COVID-19 pandemic on maternal and child mortality in low-income and middle-income countries: a modelling study.

BACKGROUND: While the COVID-19 pandemic will increase mortality due to the virus, it is also likely to increase mortality indirectly. In this study, we estimate the additional maternal and under-5 child deaths resulting from the potential disruption of health systems and decreased access to food. METHODS: We modelled three scenarios in which the coverage of essential maternal and child health interventions is reduced by 9·8-51·9% and the prevalence of wasting is increased by 10-50%. Although our scenarios are hypothetical, we sought to reflect real-world possibilities, given emerging reports of the supply-side and demand-side effects of the pandemic. We used the Lives Saved Tool to estimate the additional maternal and under-5 child deaths under each scenario, in 118 low-income and middle-income countries. We estimated additional deaths for a single month and extrapolated for 3 months, 6 months, and 12 months. FINDINGS: Our least severe scenario (coverage reductions of 9·8-18·5% and wasting increase of 10%) over 6 months would result in 253 500 additional child deaths and 12 200 additional maternal deaths. Our most severe scenario (coverage reductions of 39·3-51·9% and wasting increase of 50%) over 6 months would result in 1 157 000 additional child deaths and 56 700 additional maternal deaths. These additional deaths would represent an increase of 9·8-44·7% in under-5 child deaths per month, and an 8·3-38·6% increase in maternal deaths per month, across the 118 countries. Across our three scenarios, the reduced coverage of four childbirth interventions (parenteral administration of uterotonics, antibiotics, and anticonvulsants, and clean birth environments) would account for approximately 60% of additional maternal deaths. The increase in wasting prevalence would account for 18-23% of additional child deaths and reduced coverage of antibiotics for pneumonia and neonatal sepsis and of oral rehydration solution for diarrhoea would together account for around 41% of additional child deaths. INTERPRETATION: Our estimates are based on tentative assumptions and represent a wide range of outcomes. Nonetheless, they show that, if routine health care is disrupted and access to food is decreased (as a result of unavoidable shocks, health system collapse, or intentional choices made in responding to the pandemic), the increase in child and maternal deaths will be devastating. We hope these numbers add context as policy makers establish guidelines and allocate resources in the days and months to come. FUNDING: Bill & Melinda Gates Foundation, Global Affairs Canada.

Impact of vaccination delay on deaths averted by pneumococcal conjugate vaccine: Modeled effects in 8 country scenarios.

Delay in vaccination from schedule has been frequently documented and varies by vaccine, dose, and setting. Vaccination delay may result in the failure to prevent deaths that would have been averted by on-schedule vaccination. We constructed a model to assess the impact of delay in vaccination with pneumococcal conjugate vaccine (PCV) on under-five mortality. The model accounted for the week of age-specific risk of pneumococcal mortality, direct effect of vaccination, and herd protection. For each model run, a cohort of children were exposed to the risk of mortality and protective effect of PCV for each week of age from birth to age five. The model was run with and without vaccination delay and difference in number of deaths averted was calculated. We applied the model to eight country-specific vaccination scenarios, reflecting variations in observed vaccination delay, PCV coverage, herd effect, mortality risk, and vaccination schedule. As PCV is currently being scaled up in India, we additionally evaluated the impact of vaccination delay in India under various delay scenarios and coverage levels. We found deaths averted by PCV with and without delay to be comparable in all of the country scenarios when accounting for herd protection. In India, the greatest relative difference in deaths averted was observed at low coverage levels and greatest absolute difference was observed around 60% vaccination coverage. Under moderate delay scenarios, vaccination delay had modest impact on deaths averted by PCV in India across levels of coverage or vaccination schedule. Without accounting for herd protection, vaccination delay resulted in much greater failure to avert deaths. Our model suggests that realistic vaccination delay has a minimal impact on the number of deaths averted by PCV when accounting for herd effect. High population coverage can largely over-ride the deleterious effect of vaccination delay through herd protection.

Evaluation of methods for linking household and health care provider data to estimate effective coverage of management of child illness: results of a pilot study in Southern Province, Zambia.

BACKGROUND: Existing population-based surveys have limited accuracy for estimating the coverage and quality of management of child illness. Linking household survey data with health care provider assessments has been proposed as a means of generating more informative population-level estimates of effective coverage, but methodological issues need to be addressed. METHODS: A 2016 survey estimated effective coverage of management of child illness in Southern Province, Zambia, using multiple methods for linking temporally and geographically proximate household and health care provider data. Mothers of children <5 years were surveyed about seeking care for child illness. Information on health care providers' capacity to manage child illness, or structural quality, was assessed using case scenarios and a tool modeled on the WHO Service Availability and Readiness Assessment (SARA). Each sick child was assigned the structural quality score of their stated (exact-match) source of care. Effective coverage was calculated as the average structural quality experienced by all sick children. Children were also ecologically linked to providers using measures of geographic proximity, with and without data on non-facility providers, to assess the effects of these linking methods on effective coverage estimates. RESULTS: Data were collected on 83 providers and 385 children with fever, diarrhea, and/or symptoms of ARI in the preceding 2 weeks. Most children sought care from government facilities or community-based agents (CBAs). Effective coverage of management of child illness estimated through exact-match linking was approximately 15-points lower in each stratum than coverage of seeking skilled care due to providers' limited structural quality. Estimates generated using most measures of geographic proximity were similar to the exact-match estimate, with the exception of the kernel density estimation method in the urban area. Estimates of coverage in rural areas were greatly reduced across all methods using facility-only data if seeking care from CBAs was treated as unskilled care. CONCLUSIONS: Linking household and provider data may generate more informative estimates of effective coverage of management of child illness. Ecological linking with provider data on a sample of all skilled providers may be as effective as exact-match linking in areas with low variation in structural quality within a provider category or minimal provider bypassing.

Validity of maternal report of care-seeking for childhood illness.

BACKGROUND: Accurate data on care-seeking for child illness are needed to improve public health programs and reduce child mortality. The accuracy of maternal report of care-seeking for child illness as collected through household surveys has not been validated. METHODS: A 2016 survey compared reported care-seeking against a gold-standard of health care provider documented care-seeking events among a random sample of mothers of children <5 years in Southern Province, Zambia. Enrolled children were assigned cards with unique barcodes. Seventy-five health care providers were given smartphones with a barcode reader and instructed to scan the cards of participating children seeking care at the source, generating an electronic record of the care-seeking event. Additionally, providers gave all caregivers accessing care for a child <5 years provider-specific tokens used to verify the point of care during the household survey. Reported care-seeking events were ascertained in each household using a questionnaire modeled off the Zambia Demographic and Health Survey (DHS) / Multiple Indicator Cluster Survey (MICS). The accuracy of maternal report of care-seeking behavior was estimated by comparing care-seeking events reported by mothers against provider-documented events. RESULTS: Data were collected on 384 children with fever, diarrhea, and/or symptoms of ARI in the preceding 2 weeks. Most children sought care from government facilities or community-based agents (CBAs). We found high sensitivity (Rural: 0.91, 95% confidence interval CI 0.84-0.95; Urban: 0.98, 95% CI 0.92-0.99) and reasonable specificity (Rural: 0.71, 95% CI 0.57-0.82; Urban: 0.76, 95% CI 0.62-0.85) of maternal report of care-seeking for child illness by type of provider. Maternal report of any care-seeking and seeking care from a skilled provider had slightly higher sensitivity and specificity. Seeking care from a traditional practitioner was associated with lower odds of accurately reporting the event, while seeking care from a government provider was associated with greater odds of accurate report. The measure resulted in a slight overestimation of true care-seeking behavior in the study population. CONCLUSIONS: Maternal report is a valid measure of care-seeking for child illness in settings with high utilization of public sector providers. The study findings were limited by the low diversity in care-seeking practices for child illness and the exclusion of shops.

Validation studies for population-based intervention coverage indicators: design, analysis, and interpretation.

BACKGROUND: Population-based intervention coverage indicators are widely used to track country and program progress in improving health and to evaluate health programs. Indicator validation studies that compare survey responses to a "gold standard" measure are useful to understand whether the indicator provides accurate information. The Improving Coverage Measurement (ICM) Core Group has developed and implemented a standard approach to validating coverage indicators measured in household surveys, described in this paper. METHODS: The general design of these studies includes measurement of true health status and intervention receipt (gold standard), followed by interviews with the individuals observed, and a comparison of the observations (gold standard) to the responses to survey questions. The gold standard should use a data source external to the respondent to document need for and receipt of an intervention. Most frequently, this is accomplished through direct observation of clinical care, and/or use of a study-trained clinician to obtain a gold standard diagnosis. Follow-up interviews with respondents should employ standard survey questions, where they exist, as well as alternative or additional questions that can be compared against the standard household survey questions. RESULTS: Indicator validation studies should report on participation at every stage, and provide data on reasons for non-participation. Metrics of individual validity (sensitivity, specificity, area under the receiver operating characteristic curve) and population-level validity (inflation factor) should be reported, as well as the percent of survey responses that are "don't know" or missing. Associations between interviewer and participant characteristics and measures of validity should be assessed and reported. CONCLUSIONS: These methods allow respondent-reported coverage measures to be validated against more objective measures of need for and receipt of an intervention, and should be considered together with cognitive interviewing, discriminative validity, or reliability testing to inform decisions about which indicators to include in household surveys. Public health researchers should assess the evidence for validity of existing and proposed household survey coverage indicators and consider validation studies to fill evidence gaps.

Linking household survey and health facility data for effective coverage measures: a comparison of ecological and individual linking methods using the Multiple Indicator Cluster Survey in Côte d'Ivoire.

BACKGROUND: Population-based measures of intervention coverage are used in low- and middle-income countries for program planning, prioritization, and evaluation. There is increased interest in effective coverage, which integrates information about service quality or health outcomes. Approaches proposed for quality-adjusted effective coverage include linking data on need and service contact from population-based surveys with data on service quality from health facility surveys. However, there is limited evidence about the validity of different linking methods for effective coverage estimation. METHODS: We collaborated with the 2016 Côte d'Ivoire Multiple Indicator Cluster Survey (MICS) to link data from a health provider assessment to care-seeking data collected by the MICS in the Savanes region of Côte d'Ivoire. The provider assessment was conducted in a census of public and non-public health facilities and pharmacies in Savanes in May-June 2016. We also included community health workers managing sick children who served the clusters sampled for the MICS. The provider assessment collected information on structural and process quality for antenatal care, delivery and immediate newborn care, postnatal care, and sick child care. We linked the MICS and provider data using exact-match and ecological linking methods, including aggregate linking and geolinking methods. We compared the results obtained from exact-match and ecological methods. RESULTS: We linked 731 of 786 care-seeking episodes (93%) from the MICS to a structural quality score for the provider named by the respondent. Effective coverage estimates computed using exact-match methods were 13%-63% lower than the care-seeking estimates from the MICS. Absolute differences between exact match and ecological linking methods were ±7 percentage points for all ecological methods. Incorporating adjustments for provider category and weighting by service-specific utilization into the ecological methods generally resulted in better agreement between ecological and exact match estimates. CONCLUSIONS: Ecological linking may be a feasible and valid approach for estimating quality-adjusted effective coverage when a census of providers is used. Adjusting for provider type and caseload may improve agreement with exact match results. There remain methodological questions to be addressed to develop guidance on using linking methods for estimating quality-adjusted effective coverage, including the effect of facility sampling and time displacement.