Analysis of combined probability and nonprobability samples: A simulation evaluation and application to a teen smoking behavior survey.

In scientific studies with low-prevalence outcomes, probability sampling may be supplemented by nonprobability sampling to boost the sample size of desired subpopulation while remaining representative to the entire study population. To utilize both probability and nonprobability samples appropriately, several methods have been proposed in the literature to generate pseudo-weights, including ad-hoc weights, inclusion probability adjusted weights, and propensity score adjusted weights. We empirically compare various weighting strategies via an extensive simulation study, where probability and nonprobability samples are combined. Weight normalization and raking adjustment are also considered. Our simulation results suggest that the unity weight method (with weight normalization) and the inclusion probability adjusted weight method yield very good overall performance. This work is motivated by the Buckeye Teen Health Study, which examines risk factors for the initiation of smoking among teenage males in Ohio. To address the low response rate in the initial probability sample and low prevalence of smokers in the target population, a small convenience sample was collected as a supplement. Our proposed method yields estimates very close to the ones from the analysis using only the probability sample and enjoys the additional benefit of being able to track more teens with risky behaviors through follow-ups.

Measuring treatment coverage for neglected tropical disease control programs: analysis of a survey design.

Monitoring of treatment coverage following mass drug administration is essential to ensure program success. Coverage results reported by drug administrators are often validated by using population surveys. This study evaluates the design of a multistage cluster sample survey conducted in 2007-2008 and implemented at the district level to assess drug coverage in the 4 African countries of Burkina Faso, Ghana, Niger, and Uganda. Estimates of precision of coverage were calculated, and factors contributing to the observed variance were analyzed. Precision of ±5 percentage points was obtained in 39% (n = 12) of cases, and precision of ±10 percentage points was obtained in 77% (n = 24) of cases. The factor having the largest impact on the actual precision obtained in these surveys was the high level of clustering, the impact of which is incorporated in the design effect. Key recommendations are made for the design and analysis of future surveys; guidelines are presented for thinking through the number of clusters that should be selected and how a cluster should be designed.