Interviewer effects in food acquisition surveys.

OBJECTIVE: To understand the effects of interviewers on the responses they collect for measures of food security, income and selected survey quality measures (i.e. discrepancy between reported Supplemental Nutrition Assistance Program (SNAP) status and administrative data, length of time between initial and final interview, and missing income data) in the US Department of Agriculture's National Household Food Acquisition and Purchase Survey (FoodAPS). DESIGN: Using data from FoodAPS, multilevel models with random interviewer effects were fitted to estimate the variance in each outcome measure arising from effects of the interviewers. Covariates describing each household's socio-economic status, demographics and experience in taking the survey, and interviewer-level experience were included as fixed effects. The variance components in the outcomes due to interviewers were estimated. Outlier interviewers were profiled. SETTING: Non-institutionalized households in the continental USA (April 2012-January 2013). SUBJECTS: Individuals (n 14 317) in 4826 households who responded to FoodAPS. RESULTS: There was a substantial amount of variability in the distributions of the outcomes examined (i.e. time between initial and final interview, reported values for food security, individual income, missing income) among the FoodAPS interviewers, even after accounting for the fixed effects of the household- and interviewer-level covariates and removing extreme outlier interviewers. CONCLUSIONS: Interviewers may introduce error in food acquisition survey data when they are asked to interact with the respondents. Managers of future surveys with similarly complex data collection procedures could consider using multilevel models to adaptively identify and retrain interviewers who have extreme effects on data collection outcomes.

Video in Survey Interviews: Effects on Data Quality and Respondent Experience.

This study investigates the extent to which video technologies - now ubiquitous - might be useful for survey measurement. We compare respondents' performance and experience (n = 1,067) in live video-mediated interviews, a web survey in which prerecorded interviewers read questions, and a conventional (textual) web survey. Compared to web survey respondents, those interviewed via live video were less likely to select the same response for all statements in a battery (non-differentiation) and reported higher satisfaction with their experience but provided more rounded numerical (presumably less thoughtful) answers and selected answers that were less sensitive (more socially desirable). This suggests the presence of a live interviewer, even if mediated, can keep respondents motivated and conscientious but may introduce time pressure - a likely reason for increased rounding - and social presence - a likely reason for more socially desirable responding. Respondents "interviewed" by a prerecorded interviewer, rounded fewer numerical answers and responded more candidly than did those in the other modes, but engaged in non-differentiation more than did live video respondents, suggesting there are advantages and disadvantages for both video modes. Both live and prerecorded video seem potentially viable for use in production surveys and may be especially valuable when in-person interviews are not feasible.

A User-Driven Method for Using Research Products to Empirically Assess Item Importance in National Surveys.

Large-scale, nationally representative surveys serve many vital functions, but these surveys are often long and burdensome for respondents. Cutting survey length can help to reduce respondent burden and may improve data quality but removing items from these surveys is not a trivial matter. We propose a method to empirically assess item importance and associated burden in national surveys and guide this decision-making process using different research products produced from such surveys. This method is demonstrated using the Survey of Doctorate Recipients (SDR), a biennial survey administered to individuals with a Science, Engineering, and Health doctorate. We used three main sources of information on the SDR variables: 1) a bibliography of documents using the SDR data, 2) the SDR website that allows users to download summary data, and 3) web timing paradata and break-off rates. The bibliography was coded for SDR variable usage and citation counts. Putting this information together, we identified 35 unused items (17% of the survey) by any of these sources and found that the most burdensome items are highly important. We conclude with general recommendations for those hoping to employ similar methodologies in the future.

INTERVIEWER EFFECTS IN LIVE VIDEO AND PRERECORDED VIDEO INTERVIEWING.

Live video (LV) communication tools (e.g., Zoom) have the potential to provide survey researchers with many of the benefits of in-person interviewing, while also greatly reducing data collection costs, given that interviewers do not need to travel and make in-person visits to sampled households. The COVID-19 pandemic has exposed the vulnerability of in-person data collection to public health crises, forcing survey researchers to explore remote data collection modes-such as LV interviewing-that seem likely to yield high-quality data without in-person interaction. Given the potential benefits of these technologies, the operational and methodological aspects of video interviewing have started to receive research attention from survey methodologists. Although it is remote, video interviewing still involves respondent-interviewer interaction that introduces the possibility of interviewer effects. No research to date has evaluated this potential threat to the quality of the data collected in video interviews. This research note presents an evaluation of interviewer effects in a recent experimental study of alternative approaches to video interviewing including both LV interviewing and the use of prerecorded videos of the same interviewers asking questions embedded in a web survey ("prerecorded video" interviewing). We find little evidence of significant interviewer effects when using these two approaches, which is a promising result. We also find that when interviewer effects were present, they tended to be slightly larger in the LV approach as would be expected in light of its being an interactive approach. We conclude with a discussion of the implications of these findings for future research using video interviewing.

Respondent Driven Sampling for Immigrant Populations: A Health Survey of Foreign-Born Korean Americans.

This study examined feasibility and methodological utilities of respondent driven sampling (RDS) for Korean immigrants. We conducted the Health and Life Study of Koreans (HLSK), a Web-based RDS study targeting foreign-born Korean Americans. Through chain referrals, n = 638 participated. Geographic coverage and estimates of HLSK were compared to foreignborn Korean samples in the American Community Survey and the California Health Interview Survey as benchmarks. Compared to the benchmarks, HLSK fared well on the geographic coverage, household type and size, employment and health insurance but over-captured those who were younger, more recent immigrants, with higher education and with disability. Existing RDS-specific estimators were largely ineffective. Conclusions. RDS may serve as a cost-effective tool for recruiting recent immigrants, a harder-to-recruit subgroup within minorities. However, recruitment noncooperation posed operational challenges, a critical gap in the literature. This leaves RDS yet to be a reliable methodology.

Understanding disparities in access to naloxone among people who inject drugs in Southeast Michigan using respondent driven sampling.

BACKGROUND: Given the rising incidence of opioid overdose in the United States, naloxone access is critical for high-risk populations, such as persons who inject drugs (PWID). Yet not all PWID have access to this life-saving antidote. With PWID in Michigan recruited via respondent driven sampling in 2017, after the 2016 standing order expanding naloxone availability through local pharmacies, we explored possible access disparities. METHODS: With 46 seeds recruited from agencies serving local PWID communities, we obtained a sample of N = 410 PWID from Southeast Michigan (n = 285 form urban Detroit, and 125 for suburban/rural areas outside Detroit). Participants completed questionnaires detailing socio-demographics, health history, substance use and treatment access, including naloxone. We used multiple logistic regression to examine the predictors of self-reported naloxone access based on participant characteristics (e.g., demographics, health status) and geography (urban vs. suburban/rural). RESULTS: Self-reported naloxone access differed significantly by location (urban = 18.3 %; suburban/rural = 41.9 %). In multivariable analyses, naloxone access was significantly associated with race, household income, employment, health insurance, recent homelessness, prescription opioid usage, Hepatitis A and C status, Hepatitis A vaccination, Hepatitis C testing, access to drug treatment and services, and hospital as the usual place of care. CONCLUSION: Despite recent policies to expand access, our results indicate that naloxone access among high-risk PWID is low. This warrants future research to identify effective channels to reduce barriers and increase naloxone access.

Exploring Mechanisms of Recruitment and Recruitment Cooperation in Respondent Driven Sampling.

Respondent driven sampling (RDS) is a sampling method designed for hard-to-sample groups with strong social ties. RDS starts with a small number of arbitrarily selected participants ("seeds"). Seeds are issued recruitment coupons, which are used to recruit from their social networks. Waves of recruitment and data collection continue until reaching a sufficient sample size. Under the assumptions of random recruitment, with-replacement sampling, and a sufficient number of waves, the probability of selection for each participant converges to be proportional to their network size. With recruitment noncooperation, however, recruitment can end abruptly, causing operational difficulties with unstable sample sizes. Noncooperation may void the recruitment Markovian assumptions, leading to selection bias. Here, we consider two RDS studies: one targeting Korean immigrants in Los Angeles and in Michigan; and another study targeting persons who inject drugs in Southeast Michigan. We explore predictors of coupon redemption, associations between recruiter and recruits, and details within recruitment dynamics. While no consistent predictors of noncooperation were found, there was evidence that coupon redemption of targeted recruits was more common among those who shared social bonds with their recruiters, suggesting that noncooperation is more likely to be a feature of recruits not cooperating, rather than recruiters failing to distribute coupons.