Quantitative evaluation of an innovation contest to enhance a sexual health campaign in China.

BACKGROUND: Crowdsourcing method is an excellent tool for developing tailored interventions to improve sexual health. We evaluated the implementation of an innovation contest for sexual health promotion in China. METHODS: We organized an innovation contest over three months in 2014 for Chinese individuals < 30 years old to submit images for a sexual health promotion campaign. We solicited entries via social media and in-person events. The winning entry was adapted into a poster and distributed to STD clinics across Guangdong Province. In this study, we evaluated factors associated with images that received higher scores, described the themes of the top five finalists, and evaluated the acceptability of the winning entry using an online survey tool. RESULTS: We received 96 image submissions from 76 participants in 10 Chinese provinces. Most participants were youth (< 25 years, 85%) and non-professionals (without expertise in medicine, public health, or media, 88%). Youth were more likely to submit high-scoring entries. Images from professionals in medicine, public health, or media did not have higher scores compared to images from non-professionals. Participants were twice as likely to have learned about the contest through in-person events compared to social media. We adapted and distributed the winning entry to 300 STD clinics in 22 cities over 2 weeks. A total of 8338 people responded to an acceptability survey of the finalist entry. Among them, 79.8% endorsed or strongly endorsed being more willing to undergo STD testing after seeing the poster. CONCLUSIONS: Innovation contests may be useful for soliciting images as a part of comprehensive sexual health campaigns in low- and middle-income countries.

Ethical Concerns of and Risk Mitigation Strategies for Crowdsourcing Contests and Innovation Challenges: Scoping Review.

BACKGROUND: Crowdsourcing contests (also called innovation challenges, innovation contests, and inducement prize contests) can be used to solicit multisectoral feedback on health programs and design public health campaigns. They consist of organizing a steering committee, soliciting contributions, engaging the community, judging contributions, recognizing a subset of contributors, and sharing with the community. OBJECTIVE: This scoping review describes crowdsourcing contests by stage, examines ethical problems at each stage, and proposes potential ways of mitigating risk. METHODS: Our analysis was anchored in the specific example of a crowdsourcing contest that our team organized to solicit videos promoting condom use in China. The purpose of this contest was to create compelling 1-min videos to promote condom use. We used a scoping review to examine the existing ethical literature on crowdsourcing to help identify and frame ethical concerns at each stage. RESULTS: Crowdsourcing has a group of individuals solve a problem and then share the solution with the public. Crowdsourcing contests provide an opportunity for community engagement at each stage: organizing, soliciting, promoting, judging, recognizing, and sharing. Crowdsourcing poses several ethical concerns: organizing-potential for excluding community voices; soliciting-potential for overly narrow participation; promoting-potential for divulging confidential information; judging-potential for biased evaluation; recognizing-potential for insufficient recognition of the finalist; and sharing-potential for the solution to not be implemented or widely disseminated. CONCLUSIONS: Crowdsourcing contests can be effective and engaging public health tools but also introduce potential ethical problems. We present methods for the responsible conduct of crowdsourcing contests.

Crowdsourcing HIV Test Promotion Videos: A Noninferiority Randomized Controlled Trial in China.

BACKGROUND: Crowdsourcing, the process of shifting individual tasks to a large group, may enhance human immunodeficiency virus (HIV) testing interventions. We conducted a noninferiority, randomized controlled trial to compare first-time HIV testing rates among men who have sex with men (MSM) and transgender individuals who received a crowdsourced or a health marketing HIV test promotion video. METHODS: Seven hundred twenty-one MSM and transgender participants (≥16 years old, never before tested for HIV) were recruited through 3 Chinese MSM Web portals and randomly assigned to 1 of 2 videos. The crowdsourced video was developed using an open contest and formal transparent judging while the evidence-based health marketing video was designed by experts. Study objectives were to measure HIV test uptake within 3 weeks of watching either HIV test promotion video and cost per new HIV test and diagnosis. RESULTS: Overall, 624 of 721 (87%) participants from 31 provinces in 217 Chinese cities completed the study. HIV test uptake was similar between the crowdsourced arm (37% [114/307]) and the health marketing arm (35% [111/317]). The estimated difference between the interventions was 2.1% (95% confidence interval, -5.4% to 9.7%). Among those tested, 31% (69/225) reported a new HIV diagnosis. The crowdsourced intervention cost substantially less than the health marketing intervention per first-time HIV test (US$131 vs US$238 per person) and per new HIV diagnosis (US$415 vs US$799 per person). CONCLUSIONS: Our nationwide study demonstrates that crowdsourcing may be an effective tool for improving HIV testing messaging campaigns and could increase community engagement in health campaigns. CLINICAL TRIALS REGISTRATION: NCT02248558.

Creative Contributory Contests to Spur Innovation in Sexual Health: 2 Cases and a Guide for Implementation.

Sexual health campaigns are often designed "top-down" by public health experts, failing to engage key populations. Using the power of crowdsourcing to shape a "bottom-up" approach, this note describes 2 creative contributory contests to enhance sexual health campaigns. We provide guidance for designing creative contributory contests to improve HIV and other sexually transmitted disease testing.

Evaluating and volunteering for crowdsourced interventions: Cross-sectional data on COVID-19 safety from a University Survey.

Despite many innovative ideas generated in response to COVID-19, few studies have examined community preferences for these ideas. Our study aimed to determine university community members' preferences for three novel ideas identified through a crowdsourcing open call at the University of North Carolina (UNC) for making campus safer in the pandemic, as compared to existing (i.e. pre-COVID-19) resources. An online survey was conducted from March 30, 2021 -May 6, 2021. Survey participants included UNC students, staff, faculty, and others. The online survey was distributed using UNC's mass email listserv and research directory, departmental listservs, and student text groups. Collected data included participant demographics, COVID-19 prevention behaviors, preferences for finalist ideas vs. existing resources in three domains (graduate student supports, campus tours, and online learning), and interest in volunteering with finalist teams. In total 437 survey responses were received from 228 (52%) staff, 119 (27%) students, 78 (18%) faculty, and 12 (3%) others. Most participants were older than age 30 years (309; 71%), women (332, 78%), and white (363, 83.1%). Five participants (1%) were gender minorities, 66 (15%) identified as racial/ethnic minorities, and 46 (10%) had a disability. Most participants preferred the finalist idea for a virtual campus tour of UNC's lesser-known history compared to the existing campus tour (52.2% vs. 16.0%). For graduate student supports, 41.4% of participants indicated no preference between the finalist idea and existing supports; for online learning resources, the existing resource was preferred compared to the finalist idea (41.6% vs. 30.4%). Most participants agreed that finalists' ideas would have a positive impact on campus safety during COVID-19 (81.2%, 79.6%, and 79.2% for finalist ideas 1, 2 and 3 respectively). 61 (14.1%) participants indicated interest in volunteering with finalist teams. Together these findings contribute to the development and implementation of community-engaged crowdsourced campus safety interventions during COVID-19.

The 4 Youth by Youth (4YBY) pragmatic trial to enhance HIV self-testing uptake and sustainability: Study protocol in Nigeria.

BACKGROUND: The World Health Organization recommends HIV self-testing (HIVST) as an additional approach to HIV testing and the Nigerian government is supportive of this policy recommendation. However, effectively increasing uptake and sustainability among Nigerian youth is unknown. The goal of this study is to conduct a full-powered type I hybrid effectiveness-implementation trial to test the effectiveness of youth-friendly implementation science strategies in increasing uptake and sustainability of HIVST led by and for Nigerian youth. METHODS: Our 4 Youth by Youth (4YBY) strategy combines four core elements: 1) HIVST bundle consisting of HIVST kits and photo verification system; 2) a participatory learning community; 3) peer to peer support and technical assistance; and 4) on-site supervision and performance feedback to improve uptake and sustainability of HIVST and enhance linkage to youth-friendly health clinics for confirmatory HIV testing where needed, sexually transmitted infection (STI) testing (i.e. syphilis, gonorrhea, chlamydia, and hepatitis, STI treatment, and PrEP referral. Utilizing a stepped-wedge, cluster-randomized controlled trial, a national cohort of youth aged 14-24 recruited from 32 local government areas across 14 states and four geo-political zones in Nigeria will receive the 4YBY implementation strategy. In addition, an economic evaluation will explore the incremental cost per quality adjusted life year gained. DISCUSSION: This study will add to the limited "how-to-do it literature" on implementation science strategies in a resource-limited setting targeting youth population traditionally underrepresented in implementation science literature. Study findings will also optimize uptake and sustainability of HIVST led by and for young people themselves. TRIAL REGISTRATION: This study is registered in ClinicalTrials.govNCT04710784 (on January 15, 2021).

Assessment of a Crowdsourcing Open Call for Approaches to University Community Engagement and Strategic Planning During COVID-19.

Importance: Reimagining university life during COVID-19 requires substantial innovation and meaningful community input. One method for obtaining community input is crowdsourcing, which involves having a group of individuals work to solve a problem and then publicly share solutions. Objective: To evaluate a crowdsourcing open call as an approach to COVID-19 university community engagement and strategic planning. Design, Setting, and Participants: This qualitative study assessed a crowdsourcing open call offered from June 16 to July 16, 2020, that sought ideas to inform safety in the fall 2020 semester at the University of North Carolina at Chapel Hill (UNC). Digital methods (email and social media) were used for promotion, and submissions were collected online for 4 weeks. Participation was open to UNC students, staff, faculty, and others. Main Outcomes and Measures: Submissions were evaluated for innovation, feasibility, inclusivity, and potential to improve safety and well-being. Demographic data were collected from submitting individuals, and submissions were qualitatively analyzed for emergent themes on challenges with and solutions for addressing safety and well-being in the fall semester. Data were shared with UNC leadership to inform decision-making. Results: The open call received 82 submissions from 110 participants, including current UNC students (56 submissions [68%]), people younger than 30 years (67 [82%]), women (55 [67%]), and individuals identifying as a racial/ethnic minority or as multiracial/ethnic (49 [60%]). Seven submissions were identified as finalists and received cash prizes with the encouragement to use these funds toward idea development and implementation. Seventeen runner-up teams were linked to university resources for further development. Thematic analysis of submissions regarding challenges with the fall semester revealed not only physical health concerns and the limitations of remote learning but also challenges that have been exacerbated by the pandemic, such as a lack of mental health support, structural racism and inequality, and insufficient public transportation. Solutions included novel ideas to support mental health among specific populations (eg, graduate students and racial/ethnic minorities), improve health equity, and increase transit access. All 24 finalists and runners-up indicated interest in implementation after being notified of the open call results. Conclusions and Relevance: This study suggests that open calls are a feasible strategy for university community engagement on COVID-19, providing a stakeholder-driven approach to identifying promising ideas for enhancing safety and well-being. Open calls could be formally incorporated into university planning processes to develop COVID-19 safety strategies that are responsive to diverse community members' concerns.

Crowdsourcing in health and medical research: a systematic review.

BACKGROUND: Crowdsourcing is used increasingly in health and medical research. Crowdsourcing is the process of aggregating crowd wisdom to solve a problem. The purpose of this systematic review is to summarize quantitative evidence on crowdsourcing to improve health. METHODS: We followed Cochrane systematic review guidance and systematically searched seven databases up to September 4th 2019. Studies were included if they reported on crowdsourcing and related to health or medicine. Studies were excluded if recruitment was the only use of crowdsourcing. We determined the level of evidence associated with review findings using the GRADE approach. RESULTS: We screened 3508 citations, accessed 362 articles, and included 188 studies. Ninety-six studies examined effectiveness, 127 examined feasibility, and 37 examined cost. The most common purposes were to evaluate surgical skills (17 studies), to create sexual health messages (seven studies), and to provide layperson cardio-pulmonary resuscitation (CPR) out-of-hospital (six studies). Seventeen observational studies used crowdsourcing to evaluate surgical skills, finding that crowdsourcing evaluation was as effective as expert evaluation (low quality). Four studies used a challenge contest to solicit human immunodeficiency virus (HIV) testing promotion materials and increase HIV testing rates (moderate quality), and two of the four studies found this approach saved money. Three studies suggested that an interactive technology system increased rates of layperson initiated CPR out-of-hospital (moderate quality). However, studies analyzing crowdsourcing to evaluate surgical skills and layperson-initiated CPR were only from high-income countries. Five studies examined crowdsourcing to inform artificial intelligence projects, most often related to annotation of medical data. Crowdsourcing was evaluated using different outcomes, limiting the extent to which studies could be pooled. CONCLUSIONS: Crowdsourcing has been used to improve health in many settings. Although crowdsourcing is effective at improving behavioral outcomes, more research is needed to understand effects on clinical outcomes and costs. More research is needed on crowdsourcing as a tool to develop artificial intelligence systems in medicine. TRIAL REGISTRATION: PROSPERO: CRD42017052835. December 27, 2016.

Crowdsourcing in medical research: concepts and applications.

Crowdsourcing shifts medical research from a closed environment to an open collaboration between the public and researchers. We define crowdsourcing as an approach to problem solving which involves an organization having a large group attempt to solve a problem or part of a problem, then sharing solutions. Crowdsourcing allows large groups of individuals to participate in medical research through innovation challenges, hackathons, and related activities. The purpose of this literature review is to examine the definition, concepts, and applications of crowdsourcing in medicine. This multi-disciplinary review defines crowdsourcing for medicine, identifies conceptual antecedents (collective intelligence and open source models), and explores implications of the approach. Several critiques of crowdsourcing are also examined. Although several crowdsourcing definitions exist, there are two essential elements: (1) having a large group of individuals, including those with skills and those without skills, propose potential solutions; (2) sharing solutions through implementation or open access materials. The public can be a central force in contributing to formative, pre-clinical, and clinical research. A growing evidence base suggests that crowdsourcing in medicine can result in high-quality outcomes, broad community engagement, and more open science.

Systematic review of innovation design contests for health: spurring innovation and mass engagement.

We undertook a systematic review evaluating the effectiveness and cost of health-focused innovation design contests. We followed Cochrane guidance and systematically searched eight databases. Articles were included if they reported an open contest focused on improving health, required participants submit finished design solutions and contained a prize structure. We searched 3579 citations, identifying 29 health-focused innovation design contests which collectively received 15494 contest submissions. Contests solicited submissions worldwide (7) and exclusively from North America (13), Asia (4), Africa (2), Australia (2) and Europe (1). Submissions per contest ranged from 3 to 11354 (median of 73). Contest entrants were tasked with helping develop health promotion messages (HPM) (25) and improve predictive clinical models, protocols and/or algorithms (4). Two types of contests emerged-those focused on high-quality, innovative outcomes and those focused on the process of mass community engagement. All outcome-oriented contests had innovation design contest solutions equivalent or superior to the comparator (100%; 7/7). Two of two studies found that innovation design contests saved money (100%; 2/2). Five of seven process-oriented contests concluded the contest improved at least one health indicator (71%; 5/7). Innovation design contests are an effective way to solicit innovative solutions to health problems and spur mass community engagement.

Comparing the effectiveness of a crowdsourced video and a social marketing video in promoting condom use among Chinese men who have sex with men: a study protocol.

INTRODUCTION: Crowdsourcing has been used to spur innovation and increase community engagement in public health programmes. Crowdsourcing is the process of giving individual tasks to a large group, often involving open contests and enabled through multisectoral partnerships. Here we describe one crowdsourced video intervention in which a video promoting condom use is produced through an open contest. The aim of this study is to determine whether a crowdsourced intervention is as effective as a social marketing intervention in promoting condom use among high-risk men who have sex with men (MSM) and transgender male-to-female (TG) in China. METHOD: We evaluate videos developed by crowdsourcing and social marketing. The crowdsourcing contest involved an open call for videos. Entries were judged on capacity to promote condom use, to be shareable or 'go viral' and to give value to the individual. 1170 participants will be recruited for the randomised controlled trial. Participants need to be MSM age 16 and over who have had condomless anal sex in the last 3 months. Recruitment will be through an online banner ad on a popular MSM web page and other social media platforms. After completing an initial survey, participants will be randomly assigned to view either the social marketing video or the crowdsourcing video. Follow-up surveys will be completed at 3 weeks and 3 months after initial intervention to evaluate condomless sex and related secondary outcomes. Secondary outcomes include condom social norms, condom negotiation, condom self-efficacy, HIV/syphilis testing, frequency of sex acts and incremental cost. ETHICS AND DISSEMINATION: Approval was obtained from the ethical review boards of the Guangdong Provincial Center for Skin Diseases and STI Control, UNC and UCSF. The results of this trial will be made available through publication in peer-reviewed journals. TRIAL REGISTRATION NUMBER: NCT02516930.