Visible Cash, a Second Incentive, and Priority Mail? An Experimental Evaluation of Mailing Strategies for a Screening Questionnaire in a National Push-to-Web/Mail Survey.

In push-to-web surveys that use postal mail to contact sampled cases, participation is contingent on the mail being opened and the survey invitations being delivered. The design of the mailings is crucial to the success of the survey. We address the question of how to design invitation mailings that can grab potential respondents' attention and sway them to be interested in the survey in a short window of time. In the household screening stage of a national survey, the American Family Health Study, we experimentally tested three mailing design techniques for recruiting respondents: (1) a visible cash incentive in the initial mailing, (2) a second incentive for initial nonrespondents, and (3) use of Priority Mail in the nonresponse follow-up mailing. We evaluated the three techniques' overall effects on response rates as well as how they differentially attracted respondents with different characteristics. We found that all three techniques were useful in increasing the screening response rates, but there was little evidence that they had differential effects on sample subgroups that could help to reduce nonresponse biases.

Methods for improving participation rates in national self-administered web/mail surveys: Evidence from the United States.

In the United States, increasing access to the internet, the increasing costs of large-scale face-to-face data collections, and the general reluctance of the public to participate in intrusive in-person data collections all mean that new approaches to nationally representative surveys are urgently needed. The COVID-19 pandemic accelerated the need for faster, higher-quality alternatives to face-to-face data collection. These trends place a high priority on the evaluation of innovative web-based data collection methods that are convenient for the U.S. public and yield scientific information of high quality. The web mode is particularly appealing because it is relatively inexpensive, it is logistically flexible to implement, and it affords a high level of privacy and confidentiality when correctly implemented. With this study, we aimed to conduct a methodological evaluation of a sequential mixed-mode web/mail data collection protocol, including modular survey design concepts, which was implemented on a national probability sample in the U.S. in 2020-2021. We implemented randomized experiments to test theoretically-informed hypotheses that 1) the use of mail and increased incentives to follow up with households that did not respond to an invitation to complete a household screening questionnaire online would help to recruit different types of households; and 2) the use of modular survey design, which involves splitting a lengthy self-administered survey up into multiple parts that can be completed at a respondent's convenience, would improve survey completion rates. We find support for the use of mail and increased incentives to follow up with households that have not responded to a web-based screening questionnaire. We did not find support for the use of modular design in this context. Simple descriptive analyses also suggest that attempted telephone reminders may be helpful for the main survey.

An Experimental Evaluation of Two Approaches for Improving Response to Household Screening Efforts in National Mail/Web Surveys.

Survey researchers have carefully modified their data collection operations for various reasons, including the rising costs of data collection and the ongoing Coronavirus disease (COVID-19) pandemic, both of which have made in-person interviewing difficult. For large national surveys that require household (HH) screening to determine survey eligibility, cost-efficient screening methods that do not include in-person visits need additional evaluation and testing. A new study, known as the American Family Health Study (AFHS), recently initiated data collection with a national probability sample, using a sequential mixed-mode mail/web protocol for push-to-web US HH screening (targeting persons aged 18-49 years). To better understand optimal approaches for this type of national screening effort, we embedded two randomized experiments in the AFHS data collection. The first tested the use of bilingual respondent materials where mailed invitations to the screener were sent in both English and Spanish to 50 percent of addresses with a high predicted likelihood of having a Spanish speaker and 10 percent of all other addresses. We found that the bilingual approach did not increase the response rate of high-likelihood Spanish-speaking addresses, but consistent with prior work, it increased the proportion of eligible Hispanic respondents identified among completed screeners, especially among addresses predicted to have a high likelihood of having Spanish speakers. The second tested a form of nonresponse follow-up, where a subsample of active sampled HHs that had not yet responded to the screening invitations was sent a priority mailing with a $5 incentive, adding to the $2 incentive provided for all sampled HHs in the initial screening invitation. We found this approach to be quite valuable for increasing the screening survey response rate.

Who are we missing? Area deprivation and survey participation.

This study investigates selective participation by area deprivation in two population-based, nationally-representative surveys of children aged 5-16 years in England. We analysed the English participants (N = 22,903) in the British Child and Adolescent Mental Health Surveys, conducted in 1999 and 2004. The surveys recruited parents, teachers and children, allowing us to calculate participation rates for all three types of informant, with particularly detailed analyses possible for parents. Area-deprivation was assigned using Indices of Multiple Deprivation, 2004. We found that parents, teachers and children from more deprived areas were all substantially less likely to participate, with participation 8-9% lower in the most deprived areas than in the least deprived areas. There was substantial heterogeneity in the effect of deprivation on different types of parental non-participation, with a strong association for overall non-contact but none for overall non-cooperation. Refusal to participate in response to opt-out letters showed the opposite gradient to all other forms of non-participation, with individuals from less deprived areas being more likely to opt out. These findings indicate that the deprivation level of area of residence predicts non-response in multiple types of informants, which may exacerbate bias when estimating the prevalence of health outcomes. Furthermore, the magnitude and the direction of this may depend on the recruitment method used. Differential response by area deprivation may therefore be worth measuring in any survey.