Navigating the challenges of imposter participants in online qualitative research: lessons learned from a paediatric health services study.

BACKGROUND: The growth in online qualitative research and data collection provides several advantages for health service researchers and participants, including convenience and extended geographic reach. However, these online processes can also present unexpected challenges, including instances of participant fraud or scam behaviour. This study describes an incident of participant fraud identified during online focus group discussions and interviews for a PhD health services research project on paediatric neurodevelopmental care. METHODS: We aimed to recruit carers of Australian children with neurodevelopmental disorders. Potential participants were recruited via a publicly available social media advert on Facebook offering $50 AUD compensation. Those who expressed interest via email (n = 254) were sent a pre-interview Qualtrics survey to complete. We identified imposters at an early stage via inconsistencies in their self-reported geographical location and that captured by the survey as well as recognition of suspicious actions before, during and after focus group discussions and interviews. RESULTS: Interest in participation was unexpectedly high. We determined that all potential participants were likely imposters, posing as multiple individuals and using different IP addresses across Nigeria, Australia, and the United States. In doing so, we were able to characterise several "red flags" for identifying imposter participants, particularly those posing as multiple individuals. These comprise a combination of factors including large volumes and strange timings of email responses, unlikely demographic characteristics, short or vague interviews, a preference for nonvisual participation, fixation on monetary compensation, and inconsistencies in reported geographical location. Additionally, we propose several strategies to combat this issue such as providing proof of location or eligibility during recruitment and data collection, examining email and consent form patterns, and comparing demographic data with regional statistics. CONCLUSIONS: The emergent risk of imposter participants is an important consideration for those seeking to conduct health services research using qualitative approaches in online environments. Methodological design choices intended to improve equity and access for the target population may have an unintended consequence of improving access for fraudulent actors unless appropriate risk mitigation strategies are also employed. Lessons learned from this experience are likely to be valuable for novice health service researchers involved in online focus group discussions and interviews.

Participant Fraud in Virtual Qualitative Substance Use Research: Recommendations and Considerations for Detection and Prevention Based on a Case Study.

Background: The COVID-19 pandemic has accelerated and amplified the use of virtual research methods. While online research has several advantages, it also provides greater opportunity for individuals to misrepresent their identities to fraudulently participate in research for financial gain. Participant deception and fraud have become a growing concern for virtual research. Reports of deception and preventative strategies have been discussed within online quantitative research, particularly survey studies. Though, there is a dearth of literature surrounding these issues pertaining to qualitative studies, particularly within substance use research. Results: In this commentary, we detail an unforeseen case study of several individuals who appeared to deliberately misrepresent their identities and information during participation in a virtual synchronous qualitative substance use study. Through our experiences, we offer strategies to detect and prevent participant deception and fraud, as well as challenges to consider when implementing these approaches. Conclusions: Without general awareness and protective measures, the integrity of virtual research methods remains vulnerable to inaccuracy. As online research continues to expand, it is essential to proactively design innovative solutions to safeguard future studies against increasingly sophisticated deception and fraud.

Exploring informal caregivers' well-being during COVID-19 through online discussion forums.

BACKGROUND: COVID-19 has increased pressures on caregivers, disruptions to health services and increased health concerns during COVID-19. Reports have been made on informal carers' increased workload and limited support services during the pandemic. AIMS: This study aimed to explore how informal caregivers experienced their well-being during COVID-19 through online discussion forums. MATERIALS AND METHODS: A reflexive thematic analysis characterised by theoretical flexibility, organic inductive coding processes and theme development was conducted on online discussion forums. The method highlighted theme reviewing which was done twice to encourage data reflection. The project was conducted on a novel topic which was a new area of research interest. Semantic coding where participants' words were used directly in the interpretation and construction of themes was used. RESULTS: In the theme 'Locked in or locked away' caregivers worried about continuing care at home, due to limited freedom and worries of hiring help during a pandemic. Some expressed worries about visitation rights and grief of not being present with a loved one if they would reside in a care home. The theme 'Nothing left to give' suggested that COVID-19 exasperated caregivers' loneliness, social isolation and increased responsibilities and challenges with other roles. Bitterness, resentment and anger were felt towards lack of social support and workload. Theme 'Celebrating a virtual way of life' described how caregivers used online forums when other support services were disrupted. DISCUSSION: We discuss the role of informal caregiver that was described as all-encompassing during COVID-19. We highlight the importance of advanced planning for care home transitions and the use of online forums as a form of support. We suggest further exploration into informal caregivers' role balancing. CONCLUSION: COVID-19 seemed to affect informal caregivers negatively, but they reframed their situations and sought online support. With COVID-19-related restrictions and increased workload, COVID-19 added an all-or-nothing aspect to care home transition decisions.

[Digital information materials on dementia-an exploratory content analysis]. / Digitale Informationsmaterialen über Demenz ­ Eine explorative inhaltsanalytische Betrachtung.

BACKGROUND: A diagnosis of dementia places a heavy burden on those affected and their families. Often, difficult decisions must be made. Ideally, people with a new dementia diagnosis make informed decisions together with family members. Digital informational materials can be an important low-threshold resource for making informed decisions. They should provide comprehensive information about dementia, including both primary prevention (risk factors for the onset of dementia) and later treatment options. They should also cover precautionary measures that can make future decisions easier (e.g., advance directives, power of attorney). However, there is currently no comprehensive overview of the various online resources for dementia-related information. METHODS: This study explores digital informational materials on dementia for different target groups in the German-speaking area using an innovative systematic search strategy. It examines how these materials are structured in terms of risk factors, treatment options, and decision support. This methodological approach is new, so it is described and discussed in detail. RESULTS: The results show that most materials are intended for people with dementia and their families. Treatment measures are often discussed without explaining the risk factors they are meant to address. There is little focus on preventive measures that could support decision-making. DISCUSSION: The unbalanced presentation could lead to certain measures being difficult to understand (for laypeople) or to misinterpretation of risk factors. Important implications are drawn from these findings.

Evaluation of the effectiveness of online research groups in increasing research capacity among young African Family Physicians.

INTRODUCTION: Low research output is hindering efforts to improve health services in Sub-Saharan Africa (SSA). AfriWon Research Group of WONCA Africa has established an online collaborative research mentorship and training programme to boost research capacity among Family Physicians in SSA. This study aims to assess the effectiveness of the programme in achieving this goal. METHODOLOGY: A mixed-method descriptive cross-sectional study was used to interview the 54 members of the 2022 SOGER cohort. Structured questionnaires and key informant interviews of 12 members were used to collect data. Quantitative analysis was done using Epi Info version 7.2.5. Descriptive statistics were used to present data using frequencies and percentages. Qualitative analysis was then done by using Nvivo®. RESULTS: The 54 participants practised in 10 SSA countries and were mostly males 28 (51.9%) with a mean age of 41.56 ± 2.04 years. The majority of them 33 (61.1%) have had some form of research experience prior to joining SOGER. Members identified key benefits of the programme as fostering collaboration, mentorship and continuing education 46 (85.2%); and improvement of research skills 33 (61.1%). Barriers identified were poor collaboration and participation among members 37 (68.5%); irregular meeting schedules as a result of the different time zones 18 (33.3%); and poor internet facilities 19 (35.2%). CONCLUSION: The SOGER programme was rated as very effective in increasing the research capacity of young FPs and helping participants engage with other researchers and develop research skills. Identified areas of improvement were difficulty meeting set group targets, irregular meetings and conflicting work schedules.

Testing the relationship between lateralization on sequence-based motor tasks and language laterality using an online battery.

ABSTRACTStudies have highlighted an association between motor laterality and speech production laterality. It is thought that common demands for sequential processing may underlie this association. However, most studies in this area have relied on relatively small samples and have infrequently explored the reliability of the tools used to assess lateralization. We, therefore, established the validity and reliability of an online battery measuring sequence-based motor laterality and language laterality before exploring the associations between laterality indices on language and motor tasks. The online battery was completed by 621 participants, 52 of whom returned to complete the battery a second time. The three motor tasks included in the battery showed good between-session reliability (r ≥ .78) and were lateralized in concordance with hand preference. The novel measure of speech production laterality was left lateralized at population level as predicted, but reliability was less satisfactory (r = .62). We found no evidence of an association between sequence-based motor laterality and language laterality. Those with a left-hand preference were more strongly lateralized on motor tasks requiring midline crossing; this effect was not observed in right-handers. We conclude that there is little evidence of the co-lateralization of language and sequence-based motor skill on this battery.

Health Experiences of Women University Students of Color and Women International Students in the United States During the First Year of the COVID-19 Pandemic: Findings From a Transnational, Virtual Photovoice Study.

Women university students in the United States who are students of color and international students consistently experience health disparities. These were exacerbated due to the COVID-19 pandemic and racial upheaval following the murder of George Floyd. Within that context, this study aimed to explore the health and well-being experiences of women students of color and international women students at a university in the Midwestern United States in 2020-2021. This participatory, transnational study utilized a virtual adaptation of Photovoice with 14 participants who joined the study from five different countries. Participatory elements included participant co-researchers co-leading development of the photo "mission," icebreakers, naming of the project, design of the project logo, choosing photos for discussion and exhibits, initial analysis of emergent themes, and approval and editing of photos for exhibit. Inductive thematic analysis revealed three themes: (1) grief and loss, (2) barriers and facilitators to social health, and (3) fear and stress from simultaneous social stressors. Data was not mutually exclusive with some photos and narratives illustrating more than one theme. The women experienced grief and loss while negotiating effects of social isolation and adapting to maintain social health. Negative mental and emotional health experiences were amplified as the social context of police brutality and anti-immigrant sentiments overlaid the stress of the pandemic. Study results demonstrate need for increased and tailored health and social service support for women students of color and international women university students.

Precise display time measurement in JavaScript for web-based experiments.

Conducting research via the Internet is a formidable and ever-increasingly popular option for behavioral scientists. However, it is widely acknowledged that web-browsers are not optimized for research: In particular, the timing of display changes (e.g., a stimulus appearing on the screen), still leaves room for improvement. So far, the typically recommended best (or least bad) timing method has been a single (RAF) JavaScript function call within which one would give the display command and obtain the time of that display change. In our Study 1, we assessed two alternatives: Calling the RAF twice consecutively, or calling the RAF during a continually ongoing independent loop of recursive RAF calls. While the former has shown little or no improvement as compared to single RAF calls, with the latter we significantly and substantially improved overall precision, and achieved practically faultless precision in most practical cases. Our two basic methods for effecting display changes, plain text change and color filling, proved equally efficient. In Study 2, we reassessed the "RAF loop" timing method with image elements in combination with three different display methods: We found that the precision remained high when using either or changes - while drawing on a element consistently led to comparatively lower precision. We recommend the "RAF loop" display timing method for improved precision in future studies, and or changes when using image stimuli. We publicly share the easy-to-use code for this method, exactly as employed in our studies.

Is it ethical to use Mechanical Turk for behavioral research? Relevant data from a representative survey of MTurk participants and wages.

To understand human behavior, social scientists need people and data. In the last decade, Amazon's Mechanical Turk (MTurk) emerged as a flexible, affordable, and reliable source of human participants and was widely adopted by academics. Yet despite MTurk's utility, some have questioned whether researchers should continue using the platform on ethical grounds. The brunt of their concern is that people on MTurk are financially insecure, subject to abuse, and earn inhumane wages. We investigated these issues with two representative probability surveys of the U.S. MTurk population (N = 4094). The surveys revealed: (1) the financial situation of people on MTurk mirrors the general population, (2) most participants do not find MTurk stressful or requesters abusive, and (3) MTurk offers flexibility and benefits that most people value above other options for work. People reported it is possible to earn more than $10 per hour and said they would not trade the flexibility of MTurk for less than $25 per hour. Altogether, our data are important for assessing whether MTurk is an ethical place for research.

Frustration and ennui among Amazon MTurk workers.

Academics are increasingly turning to crowdsourcing platforms to recruit research participants. Their endeavors have benefited from a proliferation of studies attesting to the quality of crowdsourced data or offering guidance on managing specific challenges associated with doing crowdsourced research. Thus far, however, relatively little is known about what it is like to be a participant in crowdsourced research. Our analysis of almost 1400 free-text responses provides insight into the frustrations encountered by workers on one widely used crowdsourcing site: Amazon's MTurk. Some of these frustrations stem from inherent limitations of the MTurk platform and cannot easily be addressed by researchers. Many others, however, concern factors that are directly controllable by researchers and that may also be relevant for researchers using other crowdsourcing platforms such as Prolific or CrowdFlower. Based on participants' accounts of their experiences as crowdsource workers, we offer recommendations researchers might consider as they seek to design online studies that demonstrate consideration for respondents and respect for their time, effort, and dignity.

Introducing the Fear Learning and Anxiety Response (FLARe) app and web portal for the remote delivery of fear conditioning experiments.

Experimental paradigms measuring key psychological constructs can enhance our understanding of mechanisms underlying human psychological well-being and mental health. Delivering such paradigms remotely affords opportunities to reach larger, more representative samples than is typically possible with in-person research. The efficiency gained from remote delivery makes it easier to test replication of previously established effects in well-powered samples. There are several challenges to the successful development and delivery of remote experimental paradigms, including use of an appropriate delivery platform, identifying feasible outcome measures, and metrics of participant compliance. In this paper, we present FLARe (Fear Learning and Anxiety Response), open-source software in the form of a smartphone app and web portal for the creation and delivery of remote fear conditioning experiments. We describe the benefits and challenges associated with the creation of a remote delivery platform for fear conditioning, before presenting in detail the resultant software suite, and one instance of deploying this using the FLARe Research infrastructure. We provide examples of the application of FLARe to several research questions which illustrate the benefits of the remote approach to experiment delivery. The FLARe smartphone app and web portal are available for use by other researchers and have been designed to be user-friendly and intuitive. We hope that FLARe will be a useful tool for those interested in conducting well-powered fear conditioning studies to inform our understanding of the development and treatment of anxiety disorders.

Do you know the Wooly Bully? Testing era-based knowledge to verify participant age online.

People in online studies sometimes misrepresent themselves. Regardless of their motive for doing so, participant misrepresentation threatens the validity of research. Here, we propose and evaluate a way to verify the age of online respondents: a test of era-based knowledge. Across six studies (N = 1543), participants of various ages completed an age verification instrument. The instrument assessed familiarity with cultural phenomena (e.g., songs and TV shows) from decades past and present. We consistently found that our instrument discriminated between people of different ages. In Studies 1a and 1b, self-reported age correlated strongly with performance on the instrument (mean r = .8). In Study 2, the instrument reliably detected imposters who we knew were misrepresenting their age. For impostors, self-reported age did not correlate with performance on the instrument (r = .077). Finally, in Studies 3a, 3b, and 3c, the instrument remained robust with African Americans, people from low educational backgrounds, and recent immigrants to the United States. Thus, our instrument shows promise for verifying the age of online respondents, and, as we discuss, our approach of assessing "insider knowledge" holds great promise for verifying other identities within online studies.

Online eye tracking and real-time sentence processing: On opportunities and efficacy for capturing psycholinguistic effects of different magnitudes and diversity.

Online research methods have the potential to facilitate equitable accessibility to otherwise-expensive research resources, as well as to more diverse populations and language combinations than currently populate our studies. In psycholinguistics specifically, webcam-based eye tracking is emerging as a powerful online tool capable of capturing sentence processing effects in real time. The present paper asks whether webcam-based eye tracking provides the necessary granularity to replicate effects-crucially both large and small-that tracker-based eye tracking has shown. Using the Gorilla Experiment Builder platform, this study set out to replicate two psycholinguistic effects: a robust one, the verb semantic constraint effect, first reported in Altmann and Kamide,  Cognition 73(3), 247-264 (1999), and a smaller one, the lexical interference effect, first examined by Kukona et al. Journal of Experimental Psychology: Learning, Memory, and Cognition, 40(2), 326 (2014). Webcam-based eye tracking was able to replicate both effects, thus showing that its functionality is not limited to large effects. Moreover, the paper also reports two approaches to computing statistical power and discusses the differences in their outputs. Beyond discussing several important methodological, theoretical, and practical implications, we offer some further technical details and advice on how to implement webcam-based eye-tracking studies. We believe that the advent of webcam-based eye tracking, at least in respect of the visual world paradigm, will kickstart a new wave of more diverse studies with more diverse populations.

Quality-checking the new normal: trial modality in online jury decision-making research.

Objectives: We sought to examine differences between videotaped and written trial materials on verdicts, perceptions of trial parties, quality check outcomes, perceived salience of racial issues, and emotional states in a trial involving a Black or White defendant. Hypotheses: We predicted that verdicts and ratings of trial parties would be similar for those participants viewing a videotaped trial and those reading a written transcript. However, we suspected that emotional states might be heightened for those watching a video and that those reading transcripts would perform better on quality checks regarding trial content (but worse on those involving trial party characteristics, including defendant race). Method: Participants (N = 139 after removing those who did not meet our threshold for data quality) recruited from Amazon's Mechanical Turk were randomly assigned to watch a video or read a transcript of a trial involving an alleged murder of a police officer. They completed a questionnaire probing their verdict, perceptions of trial parties, perceived salience of racial issues, and emotional state, and responded to a series of quality checks. Results: Participants in the videotape condition performed significantly worse on quality checks than did those in the transcript condition. There were no significant differences between modalities in terms of verdict or perceived salience of racial issues. Some other differences emerged between conditions, however, with more positive perceptions of the pathologist and police officer in the transcript condition, and more negative emotion elicited by the trial involving a White defendant in the videotape condition only. Conclusions: There were no meaningful differences between videotaped and written trial materials in terms of outcome (verdict), but the presence of some trial party rating and emotional state differences stemming from modality epitomizes the internal/ecological validity trade-off in jury research. Our quality check results indicate that written transcripts may work better for obtaining valid data online. Regardless of modality, researchers must be diligent in crafting quality checks to ensure that participants are attending to the stimulus materials, particularly as more research shifts online.

Can everyone hear me? Reflections on the use of global online workshops for promoting inclusive knowledge generation.

Online research methods have risen in popularity over recent decades, particularly in the wake of COVID-19. We conducted five online workshops capturing the experiences of participatory health researchers in relation to power, as part of a collaborative project to develop global knowledge systems on power in participatory health research. These workshops included predominantly academic researchers working in 24 countries across Africa, Asia, Europe, and the Americas. Here, we reflect on the opportunities, limitations, and key considerations of using online workshops for knowledge generation and shared learning. The online workshop approach offers the potential for cross-continental knowledge exchange and for the amplification of global South voices. However, this study highlights the need for deeper exploration of power dynamics exposed by online platform use, particularly the 'digital divide' between academic partners and community co-researchers. Further research is needed to better understand the role of online platforms in generating more inclusive knowledge systems.

New principles and new paths needed for online research in mental health: Commentary on Burnette et al. (2021).

Online methods have become a powerful research tool, allowing us to conduct well-powered studies, to explore and replicate effects, and to recruit often rare and diverse samples. However, concerns about the validity and reliability of the data collected from some platforms have reached crescendo. In this issue, Burnette et al. (2021) describe how commonly employed protective measures such as captchas, response consistency requirements, and attention checks may no longer be sufficient to ensure high-quality data in survey-based studies on Amazon's Mechanical Turk. We echo and elaborate on these concerns, but believe that although imperfect, online research will continue to be incredibly important in driving progress in mental health science. Not all platforms or populations are well suited to every research question and so we posit that the future of online research will be much more varied, and in no small part supported by citizen scientists and those with lived experience. Whatever the medium, researchers cannot stand still; we must continuously reflect and adapt to technological advances, demographics, and motivational shifts of our participants. Online research is difficult but worthwhile.

Food for thought: Commentary on Burnette et al. (2021) "Concerns and recommendations for using Amazon MTurk for eating disorder research".

Burnette et al. aimed to validate two eating disorder symptom measures among transgender adults recruited from Mechanical Turk (MTurk). After identifying several data quality issues, Burnette et al. abandoned this aim and instead documented the issues they faced (e.g., demographic misrepresentation, repeat submissions, inconsistent responses across similar questions, failed attention checks). Consequently, Burnette et al. raised concerns about the use of MTurk for psychological research, particularly in an eating disorder context. However, we believe these claims are overstated because they arise from a single study not designed to test MTurk data quality. Further, despite claiming to go "above and beyond" current recommendations, Burnette et al. missed key screening procedures. In particular, they missed procedures known to prevent participants who use commercial data centers (i.e., server farms) to hide their true IP address and complete multiple surveys for financial gain. In this commentary, we outline key screening procedures that allow researchers to obtain quality MTurk data. We also highlight the importance of balancing efforts to increase data quality with efforts to maintain sample diversity. With appropriate screening procedures, which should be preregistered, MTurk remains a viable participant source that requires further validation in an eating disorder context.

Erratum to Peer et al. (2021) Data quality of platforms and panels for online behavioral research.

This erratum reports on a technical error that was discovered in Study 2 of Peer et al. (2021). Because of this technical error, some specific findings on participants' proclivity for dishonesty reported in the paper have been found incorrect. We detail the error, which only affected female participants, and its impact on the findings and report on the reanalyzed findings accounting for the error. The new findings do not change the conclusions provided in the paper, and show again that participants from MTurk are more likely to engage in dishonest behavior than participants from Prolific or CloudResearch.

Data quality of platforms and panels for online behavioral research.

We examine key aspects of data quality for online behavioral research between selected platforms (Amazon Mechanical Turk, CloudResearch, and Prolific) and panels (Qualtrics and Dynata). To identify the key aspects of data quality, we first engaged with the behavioral research community to discover which aspects are most critical to researchers and found that these include attention, comprehension, honesty, and reliability. We then explored differences in these data quality aspects in two studies (N ~ 4000), with or without data quality filters (approval ratings). We found considerable differences between the sites, especially in comprehension, attention, and dishonesty. In Study 1 (without filters), we found that only Prolific provided high data quality on all measures. In Study 2 (with filters), we found high data quality among CloudResearch and Prolific. MTurk showed alarmingly low data quality even with data quality filters. We also found that while reputation (approval rating) did not predict data quality, frequency and purpose of usage did, especially on MTurk: the lowest data quality came from MTurk participants who report using the site as their main source of income but spend few hours on it per week. We provide a framework for future investigation into the ever-changing nature of data quality in online research, and how the evolving set of platforms and panels performs on these key aspects.

Conversations between women with vulval lichen sclerosus: a thematic analysis of online forums.

BACKGROUND: Vulval lichen sclerosus (VLS) is a common condition. Despite this, there is a paucity of research investigating the impact on women's lives. Some women with VLS utilise online forums to discuss their priorities and concerns. This dialogue gives insight into the experiences of women living with VLS. METHODS: We identified the most popular public forums containing discussions between women with VLS. Inductive, thematic analysis was applied to 202 online posts spanning a six-year period. RESULTS: Five key themes were identified. Theme 1 pertained to difficulties with diagnosis. Women experience frequent delays and misdiagnosis. They report health care professionals (HCPs) with poor knowledge of their condition and some that were dismissive of their symptoms. Upon diagnosis women expressed relief and frustration. Theme 2 related to rationalisation and validation of their experience. Women expressed a desire to know why they were affected, what caused their symptoms and gain reassurance. Theme 3 dealt with women's motivation to control their condition. Women want to know what triggers a flare-up so they can limit their relapses. They want to self-manage their condition and have an active role in partnership with HCPs. Theme 4 related to women sharing and seeking advice from the forums. The lived experiences of other women is valued by fellow sufferers. In particular, women are keen to try other treatments, conventional and alternative. The final theme related to the social repercussions of the condition. Sociocultural factors may prevent women from talking about their condition to friends, family and HCPs. They feel embarrassed by their symptoms. Some women reported relationship breakdown as a repercussion of the disease. CONCLUSIONS: Improving the knowledge of HCPs with regards to VLS may reduce problems with diagnosis. In addition, delivering improved women's health education in schools may reduce the taboo attached to women's health. This may empower women to talk about their condition and seek help sooner. Once diagnosed, clinicians with the appropriate expertise should care for women with VLS. Women should be encouraged to take an active role in managing their condition in partnership with clinicians. Future research priorities include identifying the aetiology, triggers for flare-ups and novel therapies.