Prospective sampling bias in COVID-19 recruitment methods: experimental evidence from a national randomized survey testing recruitment materials.

BACKGROUND: In the context of the COVID-19 pandemic, social science research has required recruiting many prospective participants. Many researchers have explicitly taken advantage of widespread public interest in COVID-19 to advertise their studies. Leveraging this interest, however, risks creating unrepresentative samples due to differential interest in the topic. In this study, we investigate the design of survey recruitment materials with respect to the views of resultant participants. METHODS: Within a pan-Canadian survey (stratified random mail sampling, n = 1969), the design of recruitment invitations to prospective respondents was experimentally varied, with some prospective respondents receiving COVID-specific recruitment messages and others receiving more general recruitment messages (described as research about health and health policy). All respondents participated, however, in the same survey, allowing comparison of both demographic and attitudinal features between these groups. RESULTS: Respondents recruited via COVID-19 specific postcards were more likely to agree that COVID-19 is serious and believe that they were likely to contract COVID-19 compared to non-COVID respondents (odds = 0.71, p = 0.04; odds = 0.74, p = 0.03 respectively; comparing health to COVID-19 framed respondents). COVID-19 specific respondents were more likely to disagree that the COVID-19 threat was exaggerated compared to the non-COVID survey respondents (odds = 1.44, p = 0.02). CONCLUSIONS: COVID-19 recruitment framing garnered a higher response rate, as well as a sample with greater concern about coronavirus risks and impacts than respondents who received more neutrally framed recruitment materials.

How well do covariates perform when adjusting for sampling bias in online COVID-19 research? Insights from multiverse analyses.

COVID-19 research has relied heavily on convenience-based samples, which-though often necessary-are susceptible to important sampling biases. We begin with a theoretical overview and introduction to the dynamics that underlie sampling bias. We then empirically examine sampling bias in online COVID-19 surveys and evaluate the degree to which common statistical adjustments for demographic covariates successfully attenuate such bias. This registered study analysed responses to identical questions from three convenience and three largely representative samples (total N = 13,731) collected online in Canada within the International COVID-19 Awareness and Responses Evaluation Study ( www.icarestudy.com ). We compared samples on 11 behavioural and psychological outcomes (e.g., adherence to COVID-19 prevention measures, vaccine intentions) across three time points and employed multiverse-style analyses to examine how 512 combinations of demographic covariates (e.g., sex, age, education, income, ethnicity) impacted sampling discrepancies on these outcomes. Significant discrepancies emerged between samples on 73% of outcomes. Participants in the convenience samples held more positive thoughts towards and engaged in more COVID-19 prevention behaviours. Covariates attenuated sampling differences in only 55% of cases and increased differences in 45%. No covariate performed reliably well. Our results suggest that online convenience samples may display more positive dispositions towards COVID-19 prevention behaviours being studied than would samples drawn using more representative means. Adjusting results for demographic covariates frequently increased rather than decreased bias, suggesting that researchers should be cautious when interpreting adjusted findings. Using multiverse-style analyses as extended sensitivity analyses is recommended.

"Until I Know It's Safe for Me": The Role of Timing in COVID-19 Vaccine Decision-Making and Vaccine Hesitancy.

Managing the COVID-19 pandemic-and other communicable diseases-involves broad societal uptake of vaccines. As has been demonstrated, however, vaccine uptake is often uneven and incomplete across populations. This is a substantial challenge that must be addressed by public health efforts. To this point, significant research has focused on demographic and attitudinal correlates with vaccine hesitancy to understand uptake patterns. In this study, however, we advance understandings of individual decision-making processes involved in vaccine uptake through a mixed-methods investigation of the role of timing in COVID-19 vaccine choices. In the first step, a survey experiment, we find the timing of vaccine rollout (i.e., when a vaccine becomes available to the respondent) has a significant impact on public decision-making. Not only is there a higher level of acceptance when the vaccine becomes available at a later time, but delayed availability is correlated with both lower levels of 'desire to wait' and 'total rejection' of the vaccine. In a second step, we explore associated qualitative data, finding that temporal expressions (i.e., professing a desire to wait) can serve as a proxy for underlying non-temporal rationales, like concerns around safety, efficacy, personal situations, or altruism. By identifying these patterns, as well as the complexities of underlying factors, through a mixed-methods investigation, we can inform better vaccine-related policy and public messaging, as well as enhance our understanding of how individuals make decisions about vaccines in the context of COVID-19.

Supporting Scientific Advice through a Boundary Organization.

The complex socio-environmental issues faced by society - including climate change, resource management, and fostering resiliency in landscapes that intermix human and natural features - are difficult challenges that demand contextually appropriate evidence-based interventions. Institutional arrangements for providing scientific advice range from individual science advisors to large scientific committees or advisory councils, with a great deal of variation in their formal and informal structures. Regardless of the structuring of advisors, however, these arrangements face a common challenge: being required to speak to a wide range of issues in a time-sensitive manner, each of which has extensive stakeholder communities, deep disciplinary knowledge, and many complicating attributes. It is argued that creating a formally associated, supporting boundary organization that is tasked with supporting the advisory functions can help to resolve this challenge and improve the overall quality of advice offered. Using a case study - the California Ocean Science Trust and its advice on coastal and ocean management issues - it is argued that boundary organizations can help science advisors maintain links with disparate stakeholder communities, adjudicate between competing forms of expertise, help to provide nuance in grappling with the tensions between science and politics, and support an "honest broker" advising function.