Improving public health surveillance using a dual-frame survey of landline and cell phone numbers.

To meet challenges arising from increasing rates of noncoverage in US landline-based telephone samples due to cell-phone-only households, the Behavioral Risk Factor Surveillance System (BRFSS) expanded a traditional landline-based random digit dialing survey to a dual-frame survey of landline and cell phone numbers. In 2008, a survey of adults with cell phones only was conducted in parallel with an ongoing landline-based health survey in 18 states. The authors used the optimal approach to allocate samples into landline and cell-phone-only strata and used a new approach to weighting state-level landline and cell phone samples. They developed logistic models for each of 16 health indicators to examine whether exclusion of adults with cell phones only affected estimates after adjustment for demographic characteristics. The extents of the potential biases in landline telephone surveys that exclude cell phones were estimated. Biases resulting from exclusion of adults with cell phones only from the landline-based survey were found for 9 out of the 16 health indicators. Because landline noncoverage rates for adults with cell phones only continue to increase, these biases are likely to increase. Use of a dual-frame survey of landline and cell phone numbers assisted the BRFSS efforts in obtaining valid, reliable, and representative data.

Statistical methodology of the National Immunization Survey, 1994-2002.

OBJECTIVES: Since 1994 the National Immunization Survey (NIS) has monitored progress toward the Healthy People 2000 and 2010 vaccination goals. The NIS collects data in two phases: first, a random-digit-dialing (RDD) telephone survey to identify households with children 19-35 months old and, second, a mail survey to vaccination providers to obtain vaccination histories used to estimate vaccination coverage rates. This report reviews the methodologies used in the 1994-2002 NIS to obtain official estimates of vaccination coverage and describes the methodology used for the first three topical modules of the NIS. METHODS: From 1994 to 1997 the NIS used a variation of a two-phase estimator to compensate for missing provider-reported vaccination histories. Between 1998 and 2001 a weighting-class estimator was used. In 2002 and thereafter the weighting-class approach was refined to account for households that do not have telephones and for unvaccinated children. To collect data on immunization-related topics, the NIS sample was randomized among three topical modules: health insurance and ability to pay for vaccinations (HIM); parental knowledge and experiences about vaccinations (PKM); and daycare attendance, breastfeeding practices, and participation in the Special Supplemental Nutrition Program for Women, Infants, and Children (DCM). RESULTS: In 2001 among children with completed RDD interviews, 0.3 percent were entirely unvaccinated. Together, the new nontelephone adjustment and the refinement for unvaccinated children yielded revised estimates that were within 1.5 percentage points of the original estimates obtained using the 1998-2001 methodology. Over the six quarters during which the first three topical modules were fielded (from mid-2001 through 2002), 21,163 children were randomized to the HIM, 3576 to the PKM, and 3511 to the DCM.

When data are not missing at random: implications for measuring health conditions in the Behavioral Risk Factor Surveillance System.

OBJECTIVES: To examine the effect on estimated levels of health conditions produced from large-scale surveys, when either list-wise respondent deletion or standard demographic item-level imputation is employed. To assess the degree to which further bias reduction results from the inclusion of correlated ancillary variables in the item imputation process. DESIGN: Large cross-sectional (US level) household survey. PARTICIPANTS: 218 726 US adults (18 years and older) in the 2006 Behavioral Risk Factor Surveillance System Survey. This survey is the largest US telephone survey conducted by the Centers for Disease Control and Prevention. PRIMARY AND SECONDARY OUTCOME MEASURES: Estimated rates of severe depression among US adults. RESULTS: The use of list-wise respondent deletion and/or demographic imputation results in the underestimation of severe depression among adults in the USA. List-wise deletion produces underestimates of 9% (8.7% vs 9.5%). Demographic imputation produces underestimates of 7% (8.9% vs 9.5%). Both of these differences are significant at the 0.05 level. CONCLUSION: The use of list-wise deletion and/or demographic-only imputation may produce significant distortion in estimating national levels of certain health conditions.

Address-based versus random-digit-dial surveys: comparison of key health and risk indicators.

Use of random-digit dialing (RDD) for conducting health surveys is increasingly problematic because of declining participation rates and eroding frame coverage. Alternative survey modes and sampling frames may improve response rates and increase the validity of survey estimates. In a 2005 pilot study conducted in six states as part of the Behavioral Risk Factor Surveillance System, the authors administered a mail survey to selected household members sampled from addresses in a US Postal Service database. The authors compared estimates based on data from the completed mail surveys (n = 3,010) with those from the Behavioral Risk Factor Surveillance System telephone surveys (n = 18,780). The mail survey data appeared reasonably complete, and estimates based on data from the two survey modes were largely equivalent. Differences found, such as differences in the estimated prevalences of binge drinking (mail = 20.3%, telephone = 13.1%) or behaviors linked to human immunodeficiency virus transmission (mail = 7.1%, telephone = 4.2%), were consistent with previous research showing that, for questions about sensitive behaviors, self-administered surveys generally produce higher estimates than interviewer-administered surveys. The mail survey also provided access to cell-phone-only households and households without telephones, which cannot be reached by means of standard RDD surveys.

Implementation and applications of bootstrap methods for the National Immunization Survey.

In complex probability sample surveys, numerous adjustments are customarily made to the survey weights to reduce potential bias in survey estimates. These adjustments include sampling design (SD) weight adjustments, which account for features of the sampling plan, and non-sampling design (NSD) weight adjustments, which account for non-sampling errors and other effects. Variance estimates prepared from complex survey data customarily account for SD weight adjustments, but rarely account for all NSD weight adjustments. As a result, variance estimates may be biased and standard confidence intervals may not achieve their nominal coverage levels. We describe the implementation of the bootstrap method to account for the SD and NSD weight adjustments for complex survey data. Using data from the National Immunization Survey (NIS), we illustrate the use of the bootstrap (i). for evaluating the use of standard confidence intervals that use Taylor series approximations to variance estimators that do not account for NSD weight adjustments, (ii). for obtaining confidence intervals for ranks estimated from weighted survey data, and (iii). for evaluating the predictive power of logistic regressions using receiver operating characteristic curve analyses that account for the SD and NSD adjustments made to the survey weights.

Adjustments for non-telephone bias in random-digit-dialling surveys.

Telephone surveys are widely used in the U.S.A. for the study of health-related topics. They are subject to 'coverage bias' because they cannot sample households that do not have telephones. Although only around 5 per cent of households do not have a telephone, rates of telephone coverage show substantial variation by geography, demographic factors and socio-economic factors. In particular, lack of telephone service is more common among households that contain ethnic and racial minorities or that have lower socio-economic status with fewer opportunities for access to medical care and poorer health outcomes. Thus, failure to adequately account for households without telephones in health surveys may yield estimates of health outcomes that are misleading, particularly in states with at least moderate telephone non-coverage. The dynamic nature of the population of households without telephones offers a way of accounting for such households in telephone surveys. At any given time the population of telephone households includes households that have had a break or interruption in telephone service. Empirical results strongly suggest that these households are very similar to households that have never had telephone service. Thus, sampled households that report having had an interruption in telephone service may be used also to represent the portion of the population that has never had telephone service. This strategy can lead to a reduction in non-coverage bias in random-digit-dialling surveys. This paper presents two methods of adjusting for non-coverage of non-telephone households. The effectiveness of these methods is examined using data from the National Health Interview Survey. The interruption-in-telephone-service methods reduce non-coverage bias and can also result in a lower mean squared error. The application of the interruption-in-telephone-service methods to the National Immunization Survey is also discussed. This survey produces estimates for the 50 states and 28 urban areas. The interruption-in-telephone-service estimates tend be slightly lower than estimates resulting from poststratification and from another non-coverage adjustment method. The results suggest that the reduction in bias is greatest for variables that are highly correlated with the presence or absence of telephone service.