Cognitive Functioning and the Quality of Survey Responses: An Individual Participant Data Meta-Analysis of 10 Epidemiological Studies of Aging.

OBJECTIVES: Self-reported survey data are essential for monitoring the health and well-being of the population as it ages. For studies of aging to provide precise and unbiased results, it is necessary that the self-reported information meets high psychometric standards. In this study, we examined whether the quality of survey responses in panel studies of aging depends on respondents' cognitive abilities. METHODS: Over 17 million survey responses from 157,844 participants aged 50 years and older in 10 epidemiological studies of aging were analyzed. We derived 6 common statistical indicators of response quality from each participant's data and estimated the correlations with participants' cognitive test scores at each study wave. Effect sizes (correlations) were synthesized across studies, cognitive tests, and waves using individual participant data meta-analysis methods. RESULTS: Respondents with lower cognitive scores showed significantly more missing item responses (overall effect size ρ^ = -0.144), random measurement error (ρ^ = -0.192), Guttman errors (ρ^ = -0.233), multivariate outliers (ρ^ = -0.254), and acquiescent responses (ρ^ = -0.078); the overall effect for extreme responses (ρ^ = -0.045) was not significant. Effect sizes were consistent across studies, modes of survey administsration, and different cognitive functioning domains, although some cognitive domain specificity was also observed. DISCUSSION: Lower-quality responses among respondents with lower cognitive abilities add random and systematic errors to survey measures, reducing the reliability, validity, and reproducibility of survey study results in aging research.

Using Attributes of Survey Items to Predict Response Times May Benefit Survey Research.

Researchers have become increasingly interested in response times to survey items as a measure of cognitive effort. We used machine learning to develop a prediction model of response times based on 41 attributes of survey items (e.g., question length, response format, linguistic features) collected in a large, general population sample. The developed algorithm can be used to derive reference values for expected response times for most commonly used survey items.

Effects of Excluding Those Who Report Having "Syndomitis" or "Chekalism" on Data Quality: Longitudinal Health Survey of a Sample From Amazon's Mechanical Turk.

BACKGROUND: Researchers have implemented multiple approaches to increase data quality from existing web-based panels such as Amazon's Mechanical Turk (MTurk). OBJECTIVE: This study extends prior work by examining improvements in data quality and effects on mean estimates of health status by excluding respondents who endorse 1 or both of 2 fake health conditions ("Syndomitis" and "Chekalism"). METHODS: Survey data were collected in 2021 at baseline and 3 months later from MTurk study participants, aged 18 years or older, with an internet protocol address in the United States, and who had completed a minimum of 500 previous MTurk "human intelligence tasks." We included questions about demographic characteristics, health conditions (including the 2 fake conditions), and the Patient Reported Outcomes Measurement Information System (PROMIS)-29+2 (version 2.1) preference-based score survey. The 3-month follow-up survey was only administered to those who reported having back pain and did not endorse a fake condition at baseline. RESULTS: In total, 15% (996/6832) of the sample endorsed at least 1 of the 2 fake conditions at baseline. Those who endorsed a fake condition at baseline were more likely to identify as male, non-White, younger, report more health conditions, and take longer to complete the survey than those who did not endorse a fake condition. They also had substantially lower internal consistency reliability on the PROMIS-29+2 scales than those who did not endorse a fake condition: physical function (0.69 vs 0.89), pain interference (0.80 vs 0.94), fatigue (0.80 vs 0.92), depression (0.78 vs 0.92), anxiety (0.78 vs 0.90), sleep disturbance (-0.27 vs 0.84), ability to participate in social roles and activities (0.77 vs 0.92), and cognitive function (0.65 vs 0.77). The lack of reliability of the sleep disturbance scale for those endorsing a fake condition was because it includes both positively and negatively worded items. Those who reported a fake condition reported significantly worse self-reported health scores (except for sleep disturbance) than those who did not endorse a fake condition. Excluding those who endorsed a fake condition improved the overall mean PROMIS-29+2 (version 2.1) T-scores by 1-2 points and the PROMIS preference-based score by 0.04. Although they did not endorse a fake condition at baseline, 6% (n=59) of them endorsed at least 1 of them on the 3-month survey and they had lower PROMIS-29+2 score internal consistency reliability and worse mean scores on the 3-month survey than those who did not report having a fake condition. Based on these results, we estimate that 25% (1708/6832) of the MTurk respondents provided careless or dishonest responses. CONCLUSIONS: This study provides evidence that asking about fake health conditions can help to screen out respondents who may be dishonest or careless. We recommend this approach be used routinely in samples of members of MTurk.

Ecological risk and protective factors for food insufficiency in Los Angeles County during the COVID-19 pandemic.

OBJECTIVE: The COVID-19 pandemic increased food insufficiency: a severe form of food insecurity. Drawing on an ecological framework, we aimed to understand factors that contributed to changes in food insufficiency from April to December 2020, in a large urban population hard hit by the pandemic. DESIGN: We conducted internet surveys every 2 weeks in April-December 2020, including a subset of items from the Food Insecurity Experience Scale. Longitudinal analysis identified predictors of food insufficiency, using fixed effects models. SETTING: Los Angeles County, which has a diverse population of 10 million residents. PARTICIPANTS: A representative sample of 1535 adults in Los Angeles County who are participants in the Understanding Coronavirus in America tracking survey. RESULTS: Rates of food insufficiency spiked in the first year of the pandemic, especially among participants living in poverty, in middle adulthood and with larger households. Government food assistance from the Supplemental Nutrition Assistance Program was significantly associated with reduced food insufficiency over time, while other forms of assistance such as help from family and friends or stimulus funds were not. CONCLUSIONS: The findings highlight that during a crisis, there is value in rapidly monitoring food insufficiency and investing in government food benefits.

Behind closed doors: Protective social behavior during the COVID-19 pandemic.

The success of personal non-pharmaceutical interventions as a public health strategy requires a high level of compliance from individuals in private social settings. Strategies to increase compliance in these hard-to-reach settings depend upon a comprehensive understanding of the patterns and predictors of protective social behavior. Social cognitive models of protective behavior emphasize the contribution of individual-level factors while social-ecological models emphasize the contribution of environmental factors. This study draws on 28 waves of survey data from the Understanding Coronavirus in America survey to measure patterns of adherence to two protective social behaviors-private social-distancing behavior and private masking behavior-during the COVID-19 pandemic and to assess the role individual and environmental factors play in predicting adherence. Results show that patterns of adherence fall into three categories marked by high, moderate, and low levels of adherence, with just under half of respondents exhibiting a high level of adherence. Health beliefs emerge as the single strongest predictor of adherence. All other environmental and individual-level predictors have relatively poor predictive power or primarily indirect effects.

Role of Parents' Perceived Risk and Responsibility in Deciding on Children's COVID-19 Vaccination.

OBJECTIVES: We examined associations between parents' reports for whether their children had been vaccinated against coronavirus disease 2019 (COVID-19) and parents' perceptions of the vaccine's long-term risk, as well as their own sense of responsibility on deciding to vaccinate or not vaccinate their children. METHODS: During the period when the Omicron variant was dominant (February-March 2022), we surveyed parents from a nationally representative, probability-based Internet panel about vaccination of their school-aged children, perceptions that the vaccine's long-term risk exceeds risks without vaccination (henceforth: comparative long-term risk), their tendency to feel more responsible if their child became sick from vaccination than when unvaccinated (henceforth: anticipated responsibility), and their own vaccination status. We used multivariate analyses to assess associations of children's COVID-19 vaccination with parental comparative long-term risk perceptions, anticipated responsibility, parents' vaccination status, and demographics. RESULTS: Among 1715 parent respondents (71% of eligible), 45% perceived vaccine-related comparative long-term risk and 18% perceived greater anticipated responsibility from vaccination than no vaccination. After accounting for parental vaccination, parents who were more concerned about comparative long-term risk and who reported greater anticipated responsibility were 6% (95% confidence interval, -0.09 to -0.03; P < .001) and 15% (95% confidence interval, -0.19 to -0.11; P < .001) less likely to have vaccinated their children, respectively. Findings were driven by vaccinated parents. CONCLUSIONS: Parents' perceptions of the COVID-19 vaccine's long-term comparative risk and their greater anticipated responsibility for children getting sick if vaccinated (versus not) were associated with lower vaccine uptake among children of vaccinated parents.

Using Item Response Times in Online Questionnaires to Detect Mild Cognitive Impairment.

OBJECTIVES: With the increase in web-based data collection, response times (RTs) for survey items have become a readily available byproduct in most online studies. We examined whether RTs in online questionnaires can prospectively discriminate between cognitively normal respondents and those with cognitive impairment, no dementia (CIND). METHOD: Participants were 943 members of a nationally representative internet panel, aged 50 and older. We analyzed RTs that were passively recorded as paradata for 37 surveys (1,053 items) administered online over 6.5 years. A multilevel location-scale model derived 3 RT parameters for each survey: (1) a respondent's average RT and 2 components of intraindividual RT variability addressing (2) systematic RT adjustments and (3) unsystematic RT fluctuations. CIND status was determined at the end of the 6.5-year period. RESULTS: All 3 RT parameters were significantly associated with CIND, with a combined predictive accuracy of area under the receiver-operating characteristic curve = 0.74. Slower average RTs, smaller systematic RT adjustments, and greater unsystematic RT fluctuations prospectively predicted a greater likelihood of CIND over periods of up to 6.5, 4.5, and 1.5 years, respectively. DISCUSSION: RTs for survey items are a potential early indicator of CIND, which may enhance analyses of predictors, correlates, and consequences of cognitive impairment in online survey research.

Regular Cannabis Use During the First Year of the Pandemic: Studying Trajectories Rather Than Prevalence.

INTRODUCTION: Cannabis use in the U.S. rose early in the COVID-19 pandemic, but it is unclear whether that rise was temporary or permanent. This study estimated the nature and sociodemographic correlates of U.S. adult subpopulations regularly using cannabis by examining weekly trajectories of use during the first year of the pandemic. METHODS: Data came from the Understanding America Study, a nationally representative panel of U.S. adults (N=8,397; March 10, 2020-March 29, 2021). A growth mixture model was deployed to identify subgroups with similar regular cannabis use. Sociodemographic correlates of subgroups were examined using multinomial logistic regression. RESULTS: Four cannabis-use groups were identified. Most participants did not regularly use cannabis (no regular use; 81.7%). The other groups increased regular use until April 2020 but then diverged. Some (7.1%) decreased thereafter, whereas others (3.4%) maintained their elevated use until October 26, 2020 before decreasing. The last group (7.7%) sustained their elevated use throughout. Individuals aged between 18 and 39 years, unmarried, living in poverty, without a college degree, and with longer unemployment or underemployment spells had higher odds of being in the other groups with more weekly use than in the no-regular-use group. CONCLUSIONS: The analyses revealed population subgroups with prolonged regular cannabis use and a disproportionate concentration of socioeconomically vulnerable members of society in these subgroups. These findings elucidate important heterogeneity in the subpopulations using cannabis, highlighting the urgent need to tailor public health programs for subgroups that may have unique service needs.

Identifying Cognitive Impairment Among Older Participants in a Nationally Representative Internet Panel.

OBJECTIVES: The Health and Retirement Study Telephone Interview for Cognitive Status (HRS TICS) score and its associated Langa-Weir cutoffs are widely used as indicators of cognitive status for research purposes in population-based studies. The classification is based on in-person and phone interviews of older individuals. Our purpose was to develop a corresponding classification for web-based self-administered assessments. METHODS: Participants were 925 members of a nationally representative internet panel, all aged 50 and older. We conducted (a) a phone interview comprised of cognitive items used to construct the HRS TICS score, and (b) a web counterpart with self-administered cognitive items, while also considering (c) other already administered web-based cognitive tests and instrumental activities of daily living survey questions, all from the same respondents. RESULTS: The web-administered HRS TICS items have only modest correlations with the same phone items, although neither mode showed universally higher scores than the other. Using latent variable modeling, we created a probability of cognitive impairment score for the web-based battery that achieved good correspondence to the phone Langa-Weir classification. DISCUSSION: The results permit analyses of predictors, correlates, and consequences of cognitive impairment in web surveys where relevant cognitive test and functional abilities items are available. We discuss challenges and caveats that may affect the findings.

Born to be wild: Second-to-fourth digit length ratio and risk preferences.

The second-to-fourth digit length ratio of an individual's hand (digit ratio) is a putative biomarker for prenatal exposure to testosterone. We examine the hypothesized negative association between the digit ratio and the preference for risk taking within a large U.S. population survey. Our statistical framework provides a cardinal proxy for the true digit ratio based on ordinal digit ratio measurements and accounts for measurement error under the assumptions of Gaussianity and time-invariant true digit ratios. Our empirical findings support the hypothesis and suggest a meaningful biological basis for risk preferences.

Influence of Perceived Adolescent Vaccination Desire on Parent Decision for Adolescent COVID-19 Vaccination.

PURPOSE: The purpose of this study was to assess the influence of adolescents' desire for COVID-19 vaccination on their parents' vaccination decision for their adolescent. METHODS: We surveyed an internet-based panel of 1,051 parents of 1,519 adolescents aged 11-18 years from February to March 2021 about their adolescent's desire for COVID-19 vaccination and whether they consider this desire in their vaccination decision for the adolescent. We used multivariable Poisson regression to assess associations with parent-stated likelihood of adolescent vaccination. RESULTS: A total of 58.3% of parents reported that they and their adolescents had the same vaccination desire; similarly, 58.3% considered their adolescent's desire in their vaccination decision. These latter parents were more likely to vaccinate their adolescent than parents who did not consider their adolescent's desire (adjusted risk ratio = 1.25 [95% confidence interval = 1.05-1.50]). DISCUSSION: Most parents considered their adolescent's desire for COVID-19 vaccination. These parents were more likely to state that they will have their adolescent receive a COVID-19 vaccine.

School-Age Children's Wellbeing and School-Related Needs During the COVID-19 Pandemic.

BACKGROUND AND OBJECTIVES: The COVID-19 pandemic and related school closures may have disrupted school-related supports and services important to children's wellbeing. However, we lack national data about US children's wellbeing and family priorities for school-related services. We sought to determine 1) children's social-emotional wellbeing and 2) needs and priorities for school-based services in the 2021-2022 school year among a US sample of parents of school-aged children. METHODS: In June 2021, we surveyed 1504 parents of children enrolling in grades K-12 in the 2021-2022 school year participating in the Understanding America Study, a nationally representative probability-based Internet panel of families completing regular internet-based surveys (Response rate to this survey was 79.2%). Parents completed the Strengths and Difficulties Questionnaire and reported their needs for school-related services regarding "support getting healthcare", "mental wellness support", "food, housing, legal or transportation support", and "learning supports and enrichment." Weighted regressions examined associations among wellbeing, needs, and sociodemographic characteristics. RESULTS: Approximately one-quarter of children had deficits in hyperactivity (26.1%), one-third in peer problems (32.6%), and 40% in prosocial areas. Most parents (83.5%) reported a school-related need, with 77% reporting learning supports and enrichment needs and 57% reporting mental wellness needs. The highest priority needs were for tutoring, socialization, increased instructional time, coping with stress, and physical activity. CONCLUSIONS: US school children have high social-emotional and school-related needs. Investments in schools are urgently needed, particularly for learning supports and mental wellness, to meet the high demand for services and parents' priorities to support child health and wellbeing.

Inferring Cognitive Abilities from Response Times to Web-Administered Survey Items in a Population-Representative Sample.

Monitoring of cognitive abilities in large-scale survey research is receiving increasing attention. Conventional cognitive testing, however, is often impractical on a population level highlighting the need for alternative means of cognitive assessment. We evaluated whether response times (RTs) to online survey items could be useful to infer cognitive abilities. We analyzed >5 million survey item RTs from >6000 individuals administered over 6.5 years in an internet panel together with cognitive tests (numerical reasoning, verbal reasoning, task switching/inhibitory control). We derived measures of mean RT and intraindividual RT variability from a multilevel location-scale model as well as an expanded version that separated intraindividual RT variability into systematic RT adjustments (variation of RTs with item time intensities) and residual intraindividual RT variability (residual error in RTs). RT measures from the location-scale model showed weak associations with cognitive test scores. However, RT measures from the expanded model explained 22−26% of the variance in cognitive scores and had prospective associations with cognitive assessments over lag-periods of at least 6.5 years (mean RTs), 4.5 years (systematic RT adjustments) and 1 year (residual RT variability). Our findings suggest that RTs in online surveys may be useful for gaining information about cognitive abilities in large-scale survey research.

Subtle mistakes in self-report surveys predict future transition to dementia.

INTRODUCTION: We investigate whether indices of subtle reporting mistakes derived from responses in self-report surveys are associated with dementia risk. METHODS: We examined 13,831 participants without dementia from the prospective, population-based Health and Retirement Study (mean age 69 ± 10 years, 59% women). Participants' response patterns in 21 questionnaires were analyzed to identify implausible responses (multivariate outliers), incompatible responses (Guttman errors), acquiescent responses, random errors, and the proportion of skipped questions. Subsequent incident dementia was determined over up to 10 years of follow-up. RESULTS: During follow-up, 2074 participants developed dementia and 3717 died. Each of the survey response indices was associated with future dementia risk controlling for confounders and accounting for death as a competing risk. Stronger associations were evident for participants who were younger and cognitively normal at baseline. DISCUSSION: Mistakes in the completion of self-report surveys in longitudinal studies may be early indicators of dementia among middle-aged and older adults.

COVID-19 vaccines and mental distress.

BACKGROUND: The COVID-19 pandemic brought about large increases in mental distress. The uptake of COVID-19 vaccines is expected to significantly reduce health risks, improve economic and social outcomes, with potential benefits to mental health. PURPOSE: To examine short-term changes in mental distress following the receipt of the first dose of the COVID-19 vaccine. METHODS: Participants included 8,003 adults from the address-based sampled, nationally representative Understanding America Study (UAS), surveyed at regular intervals between March 10, 2020, and March 31, 2021 who completed at least two waves of the survey. Respondents answered questions about COVID-19 vaccine status and self-reported mental distress as measured with the four-item Patient Health Questionnaire (PHQ-4). Fixed-effects regression models were used to identify the change in PHQ-4 scores and categorical indicators of mental distress resulting from the application of the first dose of the COVID-19 vaccine. RESULTS: People who were vaccinated between December 2020 and March 2021 reported decreased mental distress levels in the surveys conducted after receiving the first dose. The fixed-effects estimates show an average effect of receiving the vaccine equivalent to 4% of the standard deviation of PHQ-4 scores (p-value<0.01), a reduction in 1 percentage point (4% reduction from the baseline level) in the probability of being at least mildly depressed, and of 0.7 percentage points (15% reduction from the baseline level) in the probability of being severely depressed (p-value = 0.06). CONCLUSIONS: Getting the first dose of COVID-19 resulted in significant improvements in mental health, beyond improvements already achieved since mental distress peaked in the spring of 2020.

Parents' Intentions and Perceptions About COVID-19 Vaccination for Their Children: Results From a National Survey.

OBJECTIVES: Assess the degree to which US parents are likely to have their children get coronavirus disease 2019 (COVID-19) vaccines and identify parental concerns about the vaccines. METHODS: In February 2021 to March 2021, we surveyed parent members of a nationally representative probability-based Internet panel of ∼9000 adults regarding their intent to have their children receive a COVID-19 vaccination, perceptions of COVID-19 vaccines for children, and trust in sources of information about COVID-19 vaccines for children. We used descriptive and multivariate analyses to evaluate parent-stated likelihood of having their children get a COVID-19 vaccine and to assess the association between likelihood of child COVID-19 vaccination and child age, parent demographics, and parental perceptions about COVID-19 vaccines. RESULTS: Altogether, 1745 parents responded (87% of eligible parents, 3759 children). Likelihood of child COVID-19 vaccination was as follows: very likely (28%), somewhat likely (18%), somewhat unlikely (9%), very unlikely (33%), and unsure (12%). The stated likelihood of child vaccination was greater among parents of older children (P < .001) as well as among parents who had a bachelor's degree or higher education (P < .001), had already received or were likely to receive a COVID-19 vaccine (P < .001), or had Democratic affiliation (P < .001); variations existed by race and ethnicity (P = .04). Parental concerns centered around vaccine safety and side effects. A key trusted source of information about COVID-19 vaccines for children was the child's doctor. CONCLUSIONS: Less than one-half of US participants report that they are likely to have their child receive a COVID-19 vaccine. Pediatric health care providers have a major role in promoting and giving COVID-19 vaccination for children.

The role of trust in the likelihood of receiving a COVID-19 vaccine: Results from a national survey.

High acceptance of coronavirus disease 2019 (COVID-19) vaccines is instrumental to ending the pandemic. Vaccine acceptance by subgroups of the population depends on their trust in COVID-19 vaccines. We surveyed a probability-based internet panel of 7832 adults from December 23, 2020-January 19, 2021 about their likelihood of getting a COVID-19 vaccine and the following domains of trust: an individual's generalized trust, trust in COVID-19 vaccine's efficacy and safety, trust in the governmental approval process and general vaccine development process for COVID-19 vaccines, trust in their physician about COVID-19, and trust in other sources about COVID-19. We included identified at-risk subgroups: healthcare workers, older adults (65-74-year-olds and ≥ 75-year-olds), frontline essential workers, other essential workers, and individuals with high-risk chronic conditions. Of 5979 respondents, only 57.4% said they were very likely or somewhat likely to get a COVID-19 vaccine. More hesitant respondents (p < 0.05) included: women, young adults (18-49 years), Blacks, individuals with lower education, those with lower income, and individuals without high-risk chronic conditions. Lack of trust in the vaccine approval and development processes explained most of the demographic variation in stated vaccination likelihood, while other domains of trust explained less variation. We conclude that hesitancy for COVID-19 vaccines is high overall and among at-risk subgroups, and hesitancy is strongly tied to trust in the vaccine approval and development processes. Building trust is critical to ending the pandemic.