Pretesting Discrete-Choice Experiments: A Guide for Researchers.

Discrete-choice experiments (DCEs) are a frequently used method to explore the preferences of patients and other decision-makers in health. Pretesting is an essential stage in the design of a high-quality choice experiment and involves engaging with representatives of the target population to improve the readability, presentation, and structure of the preference instrument. The goal of pretesting in DCEs is to improve the validity, reliability, and relevance of the survey, while decreasing sources of bias, burden, and error associated with preference elicitation, data collection, and interpretation of the data. Despite its value to inform DCE design, pretesting lacks documented good practices or clearly reported applied examples. The purpose of this paper is: (1) to define pretesting and describe the pretesting process specifically in the context of a DCE, (2) to present a practical guide and pretesting interview discussion template for researchers looking to conduct a rigorous pretest of a DCE, and (3) to provide an illustrative example of how these resources were operationalized to inform the design of a complex DCE aimed at eliciting tradeoffs between personal privacy and societal benefit in the context of a police method known as investigative genetic genealogy (IGG).

"Idealists and capitalists": ownership attitudes and preferences in genomic citizen science.

The perspectives of genomic citizen scientists on ownership of research outputs are not well understood, yet they are useful for identifying alignment of participant expectations and project practices and can help guide efforts to develop innovative tools and strategies for managing ownership claims. Here, we report findings from 52 interviews conducted in 2018 and 2019 to understand genomic citizen science stakeholders' conceptualizations of, experiences with, and preferences for ownership of research outputs. Interviewees identified four approaches for recognizing genomic citizen scientists' ownership and related credit interests in research outputs: shared governance via commons models; fractional ownership of benefits; full and creative attribution; and offensive and defensive patenting. Interviewees also agreed that the model selected by any project should at least maximize access to research outputs and, as appropriate and to the extent possible, broadly distribute rights of control and entitlements to research benefits.

Pirate Talk: Navigating Practical, Ethical, and Legal Issues Associated with Biomedical Citizen Science Interview Studies.

In citizen science, in-depth interviews have advanced the understanding of project leaders' and citizen scientists' objectives, motivations, attitudes, and concerns. The issues encountered by researchers conducting in-depth interviews in citizen science are likely not unique to this field. However, these issues can surface and play out in distinct ways that depend on the scientific and sociopolitical circumstances of citizen science communities and projects. Researchers' experiences conducting in-depth interviews are the subject of a growing literature that describes considerations for conducting research with discrete populations. We aim to contribute to this literature by describing salient practical, ethical, and legal issues to consider when interviewing biomedical citizen scientists, with a focus on bottom-up biomedical citizen scientists who have loose or no affiliations with traditional scientific institutions. These issues concern how to define the interview population; earn trust and demonstrate trustworthiness given past treatment of bottom-up biomedical citizen scientists by traditional researchers and institutions; adapt research practices to the strong culture of openness that characterizes bottom-up biomedical citizen science; and manage potential safety concerns. This essay draws on our own experiences and those of other qualitative researchers and makes suggestions for addressing these issues in ways intended to protect study integrity and demonstrate respect for participants. We also identify questions that would benefit from broad input and continued study. Our objectives in sharing these lessons learned are to support future research and to improve understanding of this exciting participatory space.

"A cohort of pirate ships": biomedical citizen scientists' attitudes toward ethical oversight.

As biomedical citizen science initiatives become more prevalent, the unique ethical issues that they raise are attracting policy attention. The ethical oversight of bottom-up biomedical citizen science projects that are designed and executed primarily or solely by members of the public is a significant concern because the federal rules that require ethical oversight of research by institutional review boards generally do not apply to such projects, creating what has been called an ethics gap. Working to close this gap, practitioners and scholars have considered new mechanisms of ethical oversight for biomedical citizen science. To date, however, participants' attitudes about ethics and oversight preferences have not been systematically examined. This information is useful to efforts to develop ethical oversight mechanisms because it provides a basis for evaluating the likely effectiveness of specific features of such mechanisms and their acceptability from the perspective of biomedical citizen scientists. Here, we report data from qualitative interviews with 35 stakeholders in bottom-up biomedical citizen science about their general ethics attitudes and preferences regarding ethical oversight. Interviewees described ten ethical priorities and endorsed oversight mechanisms that are voluntary, community-driven, and offer guidance. Conversely, interviewees rejected mechanisms that are mandatory, hierarchical, and inflexible. Applying these findings, we conclude that expert consultation and community review models appear to align well with ethical priorities and oversight preferences of many biomedical citizen scientists, although local conditions should guide the development and use of mechanisms in specific communities.