Predicting panel attrition in longitudinal HRQoL surveys during the COVID-19 pandemic in the US.

BACKGROUND: Online longitudinal surveys may be subject to potential biases due to sample attrition. This study was designed to identify potential predictors of attrition using a longitudinal panel survey collected during the COVID-19 pandemic. METHODS: Three waves of data were collected using Amazon Mechanical Turk (MTurk), an online crowd-sourced platform. For each wave, the study sample was collected by referencing a US national representative sample distribution of age, gender, and race, based on US census data. Variables included respondents' demographics, medical history, socioeconomic status, COVID-19 experience, changes of health behavior, productivity, and health-related quality of life (HRQoL). Results were compared to pre-pandemic US norms. Measures that predicted attrition at different times of the pandemic were identified via logistic regression with stepwise selection. RESULTS: 1467 of 2734 wave 1 respondents participated in wave 2 and, 964 of 2454 wave 2 respondents participated in wave 3. Younger age group, Hispanic origin (p ≤ 0.001) and higher self-rated survey difficulty (p ≤ 0.002) consistently predicted attrition in the following wave. COVID-19 experience, employment, productivity, and limited physical activities were commonly observed variables correlated with attrition with specific measures varying by time periods. From wave 1, mental health conditions, average daily hours worked (p = 0.004), and COVID-19 impact on work productivity (p < 0.001) were associated with a higher attrition rate at wave 2, additional to the aforementioned factors. From wave 2, support of social distancing (p = 0.032), being Republican (p < 0.001), and having just enough money to make ends meet (p = 0.003) were associated with predicted attrition at wave 3. CONCLUSIONS: Attrition in this longitudinal panel survey was not random. Besides commonly identified demographic factors that contribute to panel attrition, COVID-19 presented novel opportunities to address sample biases by correlating attrition with additional behavioral and HRQoL factors in a constantly evolving environment. While age, ethnicity, and survey difficulty consistently predicted attrition, other factors, such as COVID-19 experience, changes of employment, productivity, physical health, mental health, and financial situation impacted panel attrition during the pandemic at various degrees.

Are We Losing the Most Relevant Cases First? Selective Dropout in Two Longitudinal Studies of Adolescent Pornography Use.

In longitudinal studies of pornography use, selective loss of participants who may be more vulnerable to the effects of pornography than their peers is a serious concern. To explore the potential for such selective dropout, we used data from two independent large-scale panel studies of adolescents' pornography use. Of the three types of attrition-early attrition, later attrition, and gaps in participation-only the first was substantially higher among more vulnerable adolescents, compared with other participants. Panel type (online vs. classroom-based) moderated only the association between vulnerability and participation gaps, which was significant in the classroom-based but not the online panel. Overall, this study's findings point to the importance of delaying selective dropout by developing a comprehensive plan of action, for which we offer some guidelines.

Sexual Minority Women in Longitudinal Survey Research: Is Attrition a Problem?

As more longitudinal surveys collect information on sexual orientation, evaluating the quality of these data and understanding how sexual minorities engage with the survey process are increasingly important endeavors. This study constitutes the first systematic analysis of sexual orientation as a predictor of attrition from longitudinal surveys. Drawing upon the minority stress model, we developed testable hypotheses about how sexual identity and sexual identity change relate to panel attrition. These hypotheses were subsequently tested using data from two national cohorts of Australian women from the Australian Longitudinal Study on Women's Health (one born 1973-1978, n = 11,262, and one born 1989-1995, n = 16,689). In the older cohort, sexual minority women were more likely to attrit from the survey than exclusively heterosexual women-largely due to noncontact rather than noncooperation. The associations faded once sociodemographic and health-related covariates were included in the models. Further, higher rates of noncontact were observed among women who changed their sexual identity in a more same-sex-oriented direction, compared to women with a stable sexual identity. None of these associations were apparent in the younger cohort. Taken together, our results suggest that sexual minority status may be a risk factor for panel attrition among older but not younger cohorts of women and that improved efforts to locate and contact participants who are generally vulnerable could increase the retention of sexual minorities in longitudinal studies. Effect sizes were nevertheless small, suggesting that existing research on sexual orientation using longitudinal surveys is unlikely to be biased by non-random attrition of non-heterosexual individuals.

Cash vs. vouchers vs. gifts in web surveys of a mature panel study--Main effects in a long-term incentives experiment across three panel waves.

In this study, we evaluate short- and long-term effects of three different prepaid incentives: a ballpoint-pen (gift worth approximately 2 Swiss francs), a voucher (cash card worth 10 Swiss francs) and cash (a 10-Swiss-francs' banknote) on young panellists' cooperation and response rate in three waves of a mature panel study with a sequential multi-mode design (web-based online survey, CATI, and PAPI). The survey experiment involved an alternative procedure to analyse the effect of different types of prepaid incentives, taking selective attrition into account as well as considering problems related to causal inference. The subjects were students, from randomly-selected school classes, who had finished their compulsory school in 2013. The findings are clear: cash provides the strongest direct, positive effect on the overall response rate and also on the latency until response after first contact. The other incentives did not work as efficiently as did cash. Additionally, cash is the most likely to minimise social selectivity in response. Finally, cash provides the potential to convert refusals in previous waves into cooperation.

Adjust for non-ignorable panel attrition in the analysis of leaving the parental home.

Leaving the parental home is an important life event that has received significant attention in the literature. Research on this topic relies heavily on panel data; however, panel data faces the issue of serious non-ignorable panel attrition associated with leaving the parental home. This paper addresses this issue using the UK Household Longitudinal Study (UKHLS) as a case study. It introduces an adjustment procedure that combines panel gap imputation via the next observation carried backward and inverse probability weighting based on the retrieved information about leaving the parental home. The results show that this adjustment method yields more precise model estimates for leaving the parental home, and after the adjustment, the positive marginal effects of age and living with non-biological parents, as well as the negative marginal effects of Asian ethnicity and regional house prices, become more pronounced. This adjustment method has the potential to be applied to address non-ignorable panel attrition associated with other events in different panel data.

Tree-based Machine Learning Methods for Survey Research.

Predictive modeling methods from the field of machine learning have become a popular tool across various disciplines for exploring and analyzing diverse data. These methods often do not require specific prior knowledge about the functional form of the relationship under study and are able to adapt to complex non-linear and non-additive interrelations between the outcome and its predictors while focusing specifically on prediction performance. This modeling perspective is beginning to be adopted by survey researchers in order to adjust or improve various aspects of data collection and/or survey management. To facilitate this strand of research, this paper (1) provides an introduction to prominent tree-based machine learning methods, (2) reviews and discusses previous and (potential) prospective applications of tree-based supervised learning in survey research, and (3) exemplifies the usage of these techniques in the context of modeling and predicting nonresponse in panel surveys.

Sample design, sample augmentation, and estimation for Wave 2 of the NSHAP.

OBJECTIVES: The sample for the second wave (2010) of National Social Life, Health, and Aging Project (NSHAP) was designed to increase the scientific value of the Wave 1 (2005) data set by revisiting sample members 5 years after their initial interviews and augmenting this sample where possible. METHOD: There were 2 important innovations. First, the scope of the study was expanded by collecting data from coresident spouses or romantic partners. Second, to maximize the representativeness of the Wave 2 data, nonrespondents from Wave 1 were again approached for interview in the Wave 2 sample. RESULTS: The overall unconditional response rate for the Wave 2 panel was 74%; the conditional response rate of Wave 1 respondents was 89%; the conditional response rate of partners was 84%; and the conversion rate for Wave 1 nonrespondents was 26%. DISCUSSION: The inclusion of coresident partners enhanced the study by allowing the examination of how intimate, household relationships are related to health trajectories and by augmenting the size of the NSHAP sample size for this and future waves. The uncommon strategy of returning to Wave 1 nonrespondents reduced potential bias by ensuring that to the extent possible the whole of the original sample forms the basis for the field effort. NSHAP Wave 2 achieved its field objectives of consolidating the panel, recruiting their resident spouses or romantic partners, and converting a significant proportion of Wave 1 nonrespondents.