The Millennium Villages Project: a retrospective, observational, endline evaluation.

BACKGROUND: The Millennium Villages Project (MVP) was a 10 year, multisector, rural development project, initiated in 2005, operating across ten sites in ten sub-Saharan African countries to achieve the Millennium Development Goals (MDGs). In this study, we aimed to estimate the project's impact, target attainment, and on-site spending. METHODS: In this endline evaluation of the MVP, we retrospectively selected comparison villages that best matched the project villages on possible confounding variables. Cross-sectional survey data on 40 outcomes of interest were collected from both the project and the comparison villages in 2015. Using these data, as well as on-site spending data collected during the project, we estimated project impacts as differences in outcomes between the project and comparison villages; target attainment as differences between project outcomes and prespecified targets; and on-site spending as expenditures reported by communities, donors, governments, and the project. Spending data were not collected in the comparison villages. FINDINGS: Averaged across the ten project sites, we found that impact estimates for 30 of 40 outcomes were significant (95% uncertainty intervals [UIs] for these outcomes excluded zero) and favoured the project villages. In particular, substantial effects were seen in agriculture and health, in which some outcomes were roughly one SD better in the project villages than in the comparison villages. The project was estimated to have no significant impact on the consumption-based measures of poverty, but a significant favourable impact on an index of asset ownership. Impacts on nutrition and education outcomes were often inconclusive (95% UIs included zero). Averaging across outcomes within categories, the project had significant favourable impacts on agriculture, nutrition, education, child health, maternal health, HIV and malaria, and water and sanitation. A third of the targets were met in the project sites. Total on-site spending decreased from US$132 per person in the first half of the project (of which $66 was from the MVP) to $109 per person in the second half of the project (of which $25 was from the MVP). INTERPRETATION: The MVP had favourable impacts on outcomes in all MDG areas, consistent with an integrated rural development approach. The greatest effects were in agriculture and health, suggesting support for the project's emphasis on agriculture and health systems strengthening. The project conclusively met one third of its targets. FUNDING: The Open Society Foundations, the Islamic Development Bank, and the governments of Japan, South Korea, Mali, Senegal, and Uganda.

A comparison of marginal and conditional models for capture-recapture data with application to human rights violations data.

Human rights data presents challenges for capture-recapture methodology. Lists of violent acts provided by many different groups create large, sparse tables of data for which saturated models are difficult to fit and for which simple models may be misspecified. We analyze data on killings and disappearances in Casanare, Colombia during years 1998 to 2007. Our estimates differ whether we choose to model marginal reporting probabilities and odds ratios, versus modeling the full reporting pattern in a conditional (log-linear) model. With 2629 observed killings, a marginal model we consider estimates over 9000 killings, while conditional models we consider estimate 6000-7000 killings. The latter agree with previous estimates, also from a conditional model. We see a twofold difference between the high sample coverage estimate of over 10,000 killings and low sample coverage lower bound estimate of 5200 killings. We use a simulation study to compare marginal and conditional models with at most two-way interactions and sample coverage estimators. The simulation results together with model selection criteria lead us to believe the previous estimates of total killings in Casanare may have been biased downward, suggesting that the violence was worse than previously thought. Model specification is an important consideration when interpreting population estimates from capture recapture analysis and the Casanare data is a protypical example of how that manifests.

Effective classification of the prevalence of Schistosoma mansoni.

OBJECTIVE: To present an effective classification method based on the prevalence of Schistosoma mansoni in the community. METHODS: We created decision rules (defined by cut-offs for number of positive slides), which account for imperfect sensitivity, both with a simple adjustment of fixed sensitivity and with a more complex adjustment of changing sensitivity with prevalence. To reduce screening costs while maintaining accuracy, we propose a pooled classification method. To estimate sensitivity, we use the De Vlas model for worm and egg distributions. We compare the proposed method with the standard method to investigate differences in efficiency, measured by number of slides read, and accuracy, measured by probability of correct classification. RESULTS: Modelling varying sensitivity lowers the lower cut-off more significantly than the upper cut-off, correctly classifying regions as moderate rather than lower, thus receiving life-saving treatment. The classification method goes directly to classification on the basis of positive pools, avoiding having to know sensitivity to estimate prevalence. For model parameter values describing worm and egg distributions among children, the pooled method with 25 slides achieves an expected 89.9% probability of correct classification, whereas the standard method with 50 slides achieves 88.7%. CONCLUSIONS: Among children, it is more efficient and more accurate to use the pooled method for classification of S. mansoni prevalence than the current standard method.

Pooled testing for effective estimation of the prevalence of Schistosoma mansoni.

Rapid and accurate identification of the prevalence of schistosomiasis is key for control and eradication of this devastating disease. The current screening standard for intestinal schistosomiasis is the Katz-Kato method, which look for eggs on slides of fecal matter. Although work has been done to estimate prevalence using the number of eggs on a slide, the procedure is much faster if the laboratory only reports the presence or absence of eggs on each slide. To further help reduce screening costs while maintaining accuracy, we propose a pooled method for estimating prevalence. We compare it to the standard individualed method, investigating differences in efficiency, measured by the number of slides read, and accuracy, measured by mean square error of estimation. Complication is introduced by the unknown and varying sensitivity of the procedure with population prevalence. The DeVlas model for the worm and egg distributions in the population describes how test sensitivity increases with age of the epidemic, as prevalence and intensity of infection increase, making the problem fundamentally different from earlier work in pooling. Previous literature discusses varying sensitivity with the number of positive samples within a pool, known as the "dilution effect." We model both the dilution effect and varying sensitivity with population prevalence. For model parameter values suited to younger age groups, the pooled method has less than half the mean square error of the individualed method. Thus, we can use half as many slides while maintaining accuracy. Such savings might encourage more frequent measurements in regions where schistosomiasis is a serious but neglected problem.

Improving data quality and supervision of antiretroviral therapy sites in Malawi: an application of Lot Quality Assurance Sampling.

BACKGROUND: High quality program data is critical for managing, monitoring, and evaluating national HIV treatment programs. By 2009, the Malawi Ministry of Health had initiated more than 270,000 patients on HIV treatment at 377 sites. Quarterly supervision of these antiretroviral therapy (ART) sites ensures high quality care, but the time currently dedicated to exhaustive record review and data cleaning detracts from other critical components. The exhaustive record review is unlikely to be sustainable long term because of the resources required and increasing number of patients on ART. This study quantifies the current levels of data quality and evaluates Lot Quality Assurance Sampling (LQAS) as a tool to prioritize sites with low data quality, thus lowering costs while maintaining sufficient quality for program monitoring and patient care. METHODS: In January 2010, a study team joined supervision teams at 19 sites purposely selected to reflect the variety of ART sites. During the exhaustive data review, the time allocated to data cleaning and data discrepancies were documented. The team then randomly sampled 76 records from each site, recording secondary outcomes and the time required for sampling. RESULTS: At the 19 sites, only 1.2% of records had discrepancies in patient outcomes and 0.4% in treatment regimen. However, data cleaning took 28.5 hours in total, suggesting that data cleaning for all 377 ART sites would require over 350 supervision-hours quarterly. The LQAS tool accurately identified the sites with the low data quality, reduced the time for data cleaning by 70%, and allowed for reporting on secondary outcomes. CONCLUSIONS: Most sites maintained high quality records. In spite of this, data cleaning required significant amounts of time with little effect on program estimates of patient outcomes. LQAS conserves resources while maintaining sufficient data quality for program assessment and management to allow for quality patient care.