Integrating web applications into popular survey platforms for online experiments.

Research using custom-made web applications is burgeoning as scholars increasingly conduct their experiments online. We show how researchers can integrate their web applications into popular survey software such as Qualtrics in five simple steps and provide the full JavaScript code and screenshots. This procedure allows participants to seamlessly switch from Qualtrics to their web applications without leaving the survey platform. This integration has two benefits: (1) it eliminates the risk that participants inadvertently drop out of the survey while switching from the survey software to the web application and vice versa; and (2) it saves researchers the fees charged by survey companies to host an external link on the survey platform. Hence, we make it easier to conduct research on targeted (e.g., national) samples using web applications.

Early public adherence with and support for stay-at-home COVID-19 mitigation strategies despite adverse life impact: a transnational cross-sectional survey study in the United States and Australia.

BACKGROUND: Governments worldwide recommended unprecedented measures to contain the coronavirus disease 2019 (COVID-19) pandemic, caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). As pressure mounted to scale back measures, understanding public priorities was critical. We assessed initial public adherence with and support for stay-at-home orders in nations and cities with different SARS-CoV-2 infection and COVID-19 death rates. METHODS: Cross-sectional surveys were administered to representative samples of adults aged ≥18 years from regions with different SARS-CoV-2 prevalences from April 2-8, 2020. Regions included two nations [the United States (US-high prevalence) and Australia (AU-low prevalence)] and two US cities [New York City (NY-high prevalence) and Los Angeles (LA-low prevalence)]. Regional SARS-CoV-2 and COVID-19 prevalence (cumulative SARS-CoV-2 infections, COVID-19 deaths) as of April 8, 2020: US (363,321, 10,845), AU (5956, 45), NY (81,803, 4571), LA (7530, 198). Of 8718 eligible potential respondents, 5573 (response rate, 63.9%) completed surveys. Median age was 47 years (range, 18-89); 3039 (54.5%) were female. RESULTS: Of 5573 total respondents, 4560 (81.8%) reported adherence with recommended quarantine or stay-at-home policies (range of samples, 75.5-88.2%). Additionally, 29.1% of respondents screened positive for anxiety or depression symptoms (range of samples, 28.6-32.0%), with higher prevalences among those of younger age, female gender, and those in quarantine or staying at home most of the time versus those who did not report these behaviours. Despite elevated prevalences of adverse mental health symptoms and significant life disruptions, 5022 respondents (90.1%) supported government-imposed stay-at-home orders (range of samples, 88.9-93.1%). Of these, 90.8% believed orders should last at least three more weeks or until public health or government officials recommended, with support spanning the political spectrum. CONCLUSIONS: Public adherence with COVID-19 mitigation policies was highly prevalent, in both highly-affected (US, NY) and minimally-affected regions (AU, LA). Despite disruption of respondents' lives, the vast majority supported continuation of extended stay-at-home orders. Despite common support, these two countries diverged in stringent mitigation implementation, which may have contributed to subsequent outcomes. These results reveal the importance of surveillance of public support for and adherence with such policies during the COVID-19 pandemic and for future infectious disease outbreaks.

Open-source software for mouse-tracking in Qualtrics to measure category competition.

Mouse-tracking is a sophisticated tool for measuring rapid, dynamic cognitive processes in real time, particularly in experiments investigating competition between perceptual or cognitive categories. We provide user-friendly, open-source software ( https://osf.io/st2ef/ ) for designing and analyzing such experiments online using the Qualtrics survey platform. The software consists of a Qualtrics template with embedded JavaScript and CSS along with R code to clean, parse, and analyze the data. No special programming skills are required to use this software. As we discuss, this software could be readily modified for use with other online survey platforms that allow the addition of custom JavaScript. We empirically validate the provided software by benchmarking its performance on previously tested stimuli (android robot faces) in a category-competition experiment with realistic crowdsourced data collection.

Comparing Self-Reported Demographic and Sexual Behavioral Factors Among Men Who Have Sex with Men Recruited Through Mechanical Turk, Qualtrics, and a HIV/STI Clinic-Based Sample: Implications for Researchers and Providers.

Recruitment for HIV research among gay, bisexual, and other men who have sex with men (MSM) has increasingly moved to the online sphere. However, there are limited data comparing the characteristics of clinic-based respondents versus those recruited via online survey platforms. MSM were recruited from three sampling sites (STI clinic, MTurk, and Qualtrics) to participate in a survey from March 2015 to April 2016. Respondents were compared between each of the sampling sites on demographics, sexual history, substance use, and attention filter passage. Attention filter passage was high for the online sampling sites (MTurk = 93%; Qualtrics = 86%), but significantly lower for the clinic-based sampling site (72%). Clinic-based respondents were significantly more racially/ethnically diverse, reported lower income, and reported more unemployment than online respondents. Clinic-based respondents reported significantly more male sexual partners in the previous 3 months (M clinic-based = 6; MTurk = 3.6; Qualtrics = 4.5), a higher proportion of gonorrhea, chlamydia, and/or syphilis in the last year, and a greater proportion of methamphetamine use (clinic-based = 21%; MTurk = 5%), and inhaled nitrates use (clinic-based = 41%; MTurk = 11%). The clinic-based sample demonstrated more demographic diversity and a greater proportion of HIV risk behaviors when compared to the online samples, but also a relatively low attention filter passage rate. We recommend the use of attention filters across all modalities to assess response validity and urge caution with online survey engines as samples may differ demographically and behaviorally when compared to clinic-based respondents.

Promises and pitfalls of Web-based experimentation in the advance of replicable psychological science: A reply to Plant (2015).

In a recent letter, Plant (2015) reminded us that proper calibration of our laboratory experiments is important for the progress of psychological science. Therefore, carefully controlled laboratory studies are argued to be preferred over Web-based experimentation, in which timing is usually more imprecise. Here we argue that there are many situations in which the timing of Web-based experimentation is acceptable and that online experimentation provides a very useful and promising complementary toolbox to available lab-based approaches. We discuss examples in which stimulus calibration or calibration against response criteria is necessary and situations in which this is not critical. We also discuss how online labor markets, such as Amazon's Mechanical Turk, allow researchers to acquire data in more diverse populations and to test theories along more psychological dimensions. Recent methodological advances that have produced more accurate browser-based stimulus presentation are also discussed. In our view, online experimentation is one of the most promising avenues to advance replicable psychological science in the near future.

QRTEngine: An easy solution for running online reaction time experiments using Qualtrics.

Performing online behavioral research is gaining increased popularity among researchers in psychological and cognitive science. However, the currently available methods for conducting online reaction time experiments are often complicated and typically require advanced technical skills. In this article, we introduce the Qualtrics Reaction Time Engine (QRTEngine), an open-source JavaScript engine that can be embedded in the online survey development environment Qualtrics. The QRTEngine can be used to easily develop browser-based online reaction time experiments with accurate timing within current browser capabilities, and it requires only minimal programming skills. After introducing the QRTEngine, we briefly discuss how to create and distribute a Stroop task. Next, we describe a study in which we investigated the timing accuracy of the engine under different processor loads using external chronometry. Finally, we show that the QRTEngine can be used to reproduce classic behavioral effects in three reaction time paradigms: a Stroop task, an attentional blink task, and a masked-priming task. These findings demonstrate that QRTEngine can be used as a tool for conducting online behavioral research even when this requires accurate stimulus presentation times.

Acceptability, Perceptions, and Experiences Regarding Electronic Patient-Reported Outcomes After Laparoscopic Cholecystectomy: Protocol for a Mixed Methods Feasibility Study.

BACKGROUND: Patient-reported outcomes (PROs) can be defined as any report of a patient's health taken directly from the patient. Routine collection of PRO data has been shown to offer potential benefits to patient-doctor communication. Electronic forms of PRO measures (PROMs) could be more beneficial in comparison to traditional PROMs in obtaining PROs from patients. However, it is currently unclear whether the routine collection of electronic PRO data could result in better outcomes for patients undergoing laparoscopic cholecystectomy (LC). OBJECTIVE: This study aims to explore the perspectives of patients and surgeons on the use of electronic PROMs. Based on prior research, technical skill and experience level of the surgeon, long-term quality of life, patient involvement in decision-making, communication skills of the surgeon, cleanliness of the ward environment, and standards of nursing care are identified to be the most important factors for the patients. METHODS: This is a mixed methods prospective study that will collect both quantitative (survey) and qualitative (interview) data. The study has two components. The first involves the distribution of an electronic presurvey to patients who received elective LC within 48 hours of their surgery (n=80). This survey will explore the perspective of patients regarding the procedure, hospital experience, long-term outcomes, and the perceived value of using PROMs. These patients will then be followed up after 1 year and given another survey. The second component involves the distribution of the same survey and the completion of structured interviews with general surgeons (n=10). The survey will ascertain what PROs from the participants are most useful for the surgeons and the interviews will focus on how the surgeons view routine PRO collection. A convenience sampling approach will be used. Surveys will be distributed through Qualtrics and interviews will be completed on Microsoft Teams. RESULTS: Data collection began on February 14, 2023. As of February 12, 2024, 71 of 80 recruited patients have been given the presurvey. The follow-up with the patients and the general surgeon components of the study have not begun. The expected completion date of this study is in April 2025. CONCLUSIONS: Overall, this study will investigate the potential of electronic PRO collection to offer value for patients and general surgeons. This approach will ensure that patient care is investigated in a multifaceted way, offering patient-centric guidance to surgeons in their approach to care. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/57344.

Survey of COVID-19 isolation cases at a major university campus in the United States.

Objective: Baylor University established a surveillance system to assess the needs of students and faculty in isolation from SARS-CoV-2 as well as any longer-term symptoms. Participants: Overall, there were 309 responses between March 20 and May 19, 2021. Methods: A survey covering experience in isolation, symptoms, vaccination, and demographic characteristics was emailed to individuals on Day 7 of isolation; a follow-up health survey was sent 30 days later. Results: Only 9.6% of respondents reported needing assistance while in isolation. Nearly 75% of respondents experienced COVID-19 symptoms in isolation, and 31.9% had remaining symptoms after isolation. Older age, being male, and more severe symptoms were associated with longer symptom duration. Those vaccinated had lower odds of developing symptoms and having symptoms remaining post-isolation. Conclusions: The present study adds to our understanding of long-COVID in young adult populations, while providing a framework for similar institutions to sustain operations during a global pandemic.

Assessing Barriers to Implementation of Machine Learning and Artificial Intelligence-Based Tools in Critical Care: Web-Based Survey Study.

BACKGROUND: Although there is considerable interest in machine learning (ML) and artificial intelligence (AI) in critical care, the implementation of effective algorithms into practice has been limited. OBJECTIVE: We sought to understand physician perspectives of a novel intubation prediction tool. Further, we sought to understand health care provider and nonprovider perspectives on the use of ML in health care. We aim to use the data gathered to elucidate implementation barriers and determinants of this intubation prediction tool, as well as ML/AI-based algorithms in critical care and health care in general. METHODS: We developed 2 anonymous surveys in Qualtrics, 1 single-center survey distributed to 99 critical care physicians via email, and 1 social media survey distributed via Facebook and Twitter with branching logic to tailor questions for providers and nonproviders. The surveys included a mixture of categorical, Likert scale, and free-text items. Likert scale means with SD were reported from 1 to 5. We used student t tests to examine the differences between groups. In addition, Likert scale responses were converted into 3 categories, and percentage values were reported in order to demonstrate the distribution of responses. Qualitative free-text responses were reviewed by a member of the study team to determine validity, and content analysis was performed to determine common themes in responses. RESULTS: Out of 99 critical care physicians, 47 (48%) completed the single-center survey. Perceived knowledge of ML was low with a mean Likert score of 2.4 out of 5 (SD 0.96), with 7.5% of respondents rating their knowledge as a 4 or 5. The willingness to use the ML-based algorithm was 3.32 out of 5 (SD 0.95), with 75% of respondents answering 3 out of 5. The social media survey had 770 total responses with 605 (79%) providers and 165 (21%) nonproviders. We found no difference in providers' perceived knowledge based on level of experience in either survey. We found that nonproviders had significantly less perceived knowledge of ML (mean 3.04 out of 5, SD 1.53 vs mean 3.43, SD 0.941; P<.001) and comfort with ML (mean 3.28 out of 5, SD 1.02 vs mean 3.53, SD 0.935; P=.004) than providers. Free-text responses revealed multiple shared concerns, including accuracy/reliability, data bias, patient safety, and privacy/security risks. CONCLUSIONS: These data suggest that providers and nonproviders have positive perceptions of ML-based tools, and that a tool to predict the need for intubation would be of interest to critical care providers. There were many shared concerns about ML/AI in health care elucidated by the surveys. These results provide a baseline evaluation of implementation barriers and determinants of ML/AI-based tools that will be important in their optimal implementation and adoption in the critical care setting and health care in general.

Changes in Food Security, Healthfulness, and Access During the Coronavirus Disease 2019 Pandemic: Results From a National United States Survey.

Background: Coronavirus disease 2019 (COVID-19) disrupted access to food and adequate nutrition and the types of foods consumed. However, little empiric data exists on the changes in American's food and nutrition habits 2 y into the pandemic. Objectives: To assess current and altered food choices ∼2 y into the COVID-19 pandemic in the months after historic public pandemic relief. Methods: A national sample of 1878 United States adults balanced by age, sex, race/ethnicity, and income completed a one-time, online, semi-quantitative, 44-item questionnaire in Fall 2021 asking about the demographics, COVID-19 food choice changes (including free-text), and consumer priorities. This analysis investigates COVID-19 impacts on food security, healthfulness, and access. Results: More than 35% of respondents reported improved food security and >45% reported improved food healthfulness compared with prepandemic status. Improvement was reported in more than 30% of Black/African-American and Hispanic/Latinx adults, adults with lower annual income, and female sex, despite over 75% reporting reduced choice of where to eat or buy food. The pandemic offered occasion for many to improve diet, but a similar number expressed that the pandemic destabilized healthy habits. Conclusions: Our novel findings suggest that by late 2021, most Americans had improved food security and food choice healthfulness, despite reduced access to food service and retail, although with worsening among a meaningful proportion of Americans as well as heterogeneity in these changes. Vigorous federal, state, city, and community responses to the pandemic may have played a role in improving the food security and food choice healthfulness during the COVID-19 pandemic. Health crises differently impact health behaviors, but when accompanied by vigorous civic and community response, food security, and food healthfulness can be fortified.

Management of endocrine surgical disorders during COVID-19 pandemic: expert opinion for non-surgical options.

PURPOSE: The COVID-19 pandemic brought unprecedented conditions for overall health care systems by restricting resources for non-COVID-19 patients. As the burden of the disease escalates, routine elective surgeries are being cancelled. The aim of this paper was to provide a guideline for management of endocrine surgical disorders during a pandemic. METHODS: We used Delphi method with a nine-scale Likert scale on two rounds of voting involving 64 experienced eminent surgeons and endocrinologists who had the necessary experience to provide insight on endocrine disorder management. All voting was done by email using a standard questionnaire. RESULTS: Overall, 37 recommendations were voted on. In two rounds, all recommendations reached an agreement and were either endorsed or rejected. Endorsed statements include dietary change in primary hyperparathyroidism, Cinacalcet treatment in secondary hyperparathyroidism, alpha-blocker administration for pheochromocytoma, methimazole ± ß-blocker combination for Graves' disease, and follow-up for fine-needle aspiration results of thyroid nodules indicated as Bethesda 3-4 cytological results and papillary microcarcinoma. CONCLUSION: This survey summarizes expert opinion for the management of endocrine surgical conditions during unprecedented times when access to surgical treatment is severely disrupted. The statements are not applicable in circumstances in which surgical treatment is possible.

The Impact of SARS-CoV-2 (COVID-19) on the Acuity of Mental Health-Related Diagnosis at Admission for Young Adults in New York City and Washington, DC: Observational Study.

BACKGROUND: The COVID-19 pandemic has required restrictive measures to mitigate transmission of the virus. Evidence has demonstrated increased generalized anxiety and depression among young adults due to the COVID-19 pandemic. However, minimal research has examined the longitudinal effect of COVID-19 over the course of time and its impact on anxiety and depression. Additionally, age and gender have been found to play a significant role on individuals' mental health, with young adults and women particularly at risk. OBJECTIVE: The aim of this study was to examine the impact of the COVID-19 pandemic on anxiety and depression upon admissions to treatment. METHODS: This was an observational study that was completed longitudinally in which the grouping variable split the time interval into five equal groups for assessments over each period of time. A total of 112 young adults (aged 18-25 years) were recruited for the study. Participants completed assessments online through a Qualtrics link. RESULTS: Psychometric properties of the admission assessments were uniformly highly statistically significant. There was a significant difference in generalized anxiety between the group-1 and group-3 time intervals. No significant difference was found across the time intervals for depression. Differences in predicting the impact of the psychometrics scores were found with respect to gender. Only the ability to participate and the quality-of-life subfactor of the Functional Assessment of Chronic Illness Therapy (FACIT) assessment were significant. CONCLUSIONS: This study sought to understand the impact that COVID-19 has had on young adults seeking mental health services during the pandemic. Gender emerged as a clear significant factor contributing to increased anxiety in young adults seeking mental health services during the pandemic. These findings have critical importance to ensuring the potential treatment success rate of clients, while providing an overarching understanding of the impact of the pandemic and establishing clinical recommendations for the treatment of individuals who are seeking out treatment.

The Social Exclusion Bench Tool (SEBT): A visual way of assessing interpersonal social exclusion.

People usually prefer to appear with an inclusive and positive attitude to others' eyes. For this reason, the self-report scales assessing social exclusion intentions are often biased by social desirability. In this work, we present an innovative graphical tool, named Social Exclusion Bench Tool (SEBT), for assessing social exclusion not influenced by social desirability. The tool is based on the consistency between social distance and physical distance evaluation. The results showed that in two samples of adults from Italy (N = 252) and the UK (N = 254), the SEBT positively correlated with self-report measures of social exclusion, but not with the social desirability measure. The tool has been preliminarily evaluated in the context of social exclusion toward migrant people, but it appears a promising instrument for assessing social exclusion intentions toward different social groups.•The self-report scales assessing social exclusion intentions are often biased by social desirability.•The Social Exclusion Bench Tool (SEBT) is an innovative visual instrument for assessing social exclusion that seems not to be influenced by social desirability.•The tool appears a promising instrument for assessing social exclusion intentions toward different social groups.

You Don't Need an App-Conducting Mobile Smoking Research Using a Qualtrics-Based Approach.

With the increasing availability of smartphones, many tobacco researchers are exploring smartphone-delivered mobile smoking interventions as a disseminable means of treatment. Most effort has been focused on the development of smartphone applications (apps) to conduct mobile smoking research to implement and validate these interventions. However, developing project-specific smartphone apps that work across multiple mobile platforms (e.g., iOS and Android) can be costly and time-consuming. Here, using a hypothetical study, we present an alternate approach to demonstrate how mobile smoking cessation and outcome evaluation can be conducted without the need of a dedicated app. Our approach uses the Qualtrics platform, a popular online survey host that is used under license by many academic institutions. This platform allows researchers to conduct device-agnostic screening, consenting, and administration of questionnaires through Qualtrics's native survey engine. Researchers can also collect ecological momentary assessment data using text messaging prompts with the incorporation of Amazon Web Services' Pinpoint. Besides these assessment capabilities, Qualtrics has the potential for delivering personalized behavioral interventions through the use of JavaScript code. By customizing the question's web elements in Qualtrics (e.g., using texts, images, videos, and buttons), researchers can integrate interactive web-based interventions and complicated behavioral and cognitive tasks into the survey. In conclusion, this Qualtrics-based methodology represents a novel and cost-effective approach for conducting mobile smoking cessation and assessment research.

Vaccinomics: a cross-sectional survey of public values.

Objective: We characterize public values regarding vaccinomics, which aims to improve vaccine safety and effectiveness using genomics.Methods: Panel survey (2020) of ≥18-year-olds with embedded animation introduced vaccinomics. Sociodemographic, health, and vaccination-related items were adapted from validated scales. Novel items measured trust in public health authorities, vaccinomics-related values, and preferences for federal funding: vaccinomics compared with vaccine issues and chronic diseases. Beginning and end of survey confidence in vaccine safety was measured to assess potential changes. Data were weighted to the U.S. Census. Vaccinomics-related concerns were stratified by sociodemographic characteristics, vaccine hesitancy status (composite outcome), reported serious vaccine reactions, and trust in public health authorities (PHA). Log binomial regression models estimated associations between these variables and agency to make vaccine-related decisions.Results: Most (70.7%, N = 1,925) respondents expected vaccinomics would increase their vaccine confidence compared to now. Agreement was highest among those without serious vaccine reaction experience (unexperienced: 74.2% versus experienced: 62.3%), with high trust in PHA (high: 83.3% versus low: 57.4%), and low vaccine hesitancy among parents of teenagers (low: 78.8% versus high: 62.5%) and adults without minor children (low: 79.8% versus high: 60.6%; all p < .01). Belief that vaccination was an individual's choice was associated with reported serious reactions (adjusted Prevalence Ratio (aPR): 1.16; 95% CI: 1.07, 1.25) and low trust (aPR: 0.91; 0.84, 0.98). Beginning versus end of survey vaccine safety perceptions were similar.Conclusion: Federal funding, communications, and policies should assure the public that vaccinomics will not remove their decision-making power and engender trust in PHA.

The student-centered classroom: the new gut feeling.

A student-centered, interactive course-based undergraduate research experience (CURE) was implemented in a microbiology course in order to provide an authentic research experience and to stimulate student interest and improve understanding of fermentation, probiotics, the human microbiome and related topics. Students were immersed in the scientific process as they used fundamental techniques to investigate the probiotic composition of a fermented milk beverage, kefir-an unknown question with no predetermined outcomes. In order to assess the benefits and effect of this learning experience on the students, pre- and post-study surveys were administered using Qualtrics. Post-study, 93% of participants agreed that fermented foods are beneficial to human health (compared to 52% pre-study), and notably, 100% of participants indicated that they plan to apply this material in both their personal and professional lives and would suggest consuming probiotics or fermented products to alleviate gastrointestinal issues. As evidenced by demographic data, this CURE is suitable for implementation at both large and small institutions with diverse student populations. Collectively, these data indicate that this collaborative, discovery-based learning experience is a powerful educational tool, encouraging students to make real-life connections between microbiology, medicine and their own health.