Practical recommendations from a multi-perspective needs and challenges assessment of citizen science games.

Citizen science games are an increasingly popular form of citizen science, in which volunteer participants engage in scientific research while playing a game. Their success depends on a diverse set of stakeholders working together-scientists, volunteers, and game developers. Yet the potential needs of these stakeholder groups and their possible tensions are poorly understood. To identify these needs and possible tensions, we conducted a qualitative data analysis of two years of ethnographic research and 57 interviews with stakeholders from 10 citizen science games, following a combination of grounded theory and reflexive thematic analysis. We identify individual stakeholder needs as well as important barriers to citizen science game success. These include the ambiguous allocation of developer roles, limited resources and funding dependencies, the need for a citizen science game community, and science-game tensions. We derive recommendations for addressing these barriers.

Signaligner Pro: A Tool to Explore and Annotate Multi-day Raw Accelerometer Data.

Human activity recognition using wearable accelerometers can enable in-situ detection of physical activities to support novel human-computer interfaces. Many of the machine-learning-based activity recognition algorithms require multi-person, multi-day, carefully annotated training data with precisely marked start and end times of the activities of interest. To date, there is a dearth of usable tools that enable researchers to conveniently visualize and annotate multiple days of high-sampling-rate raw accelerometer data. Thus, we developed Signaligner Pro, an interactive tool to enable researchers to conveniently explore and annotate multi-day high-sampling rate raw accelerometer data. The tool visualizes high-sampling-rate raw data and time-stamped annotations generated by existing activity recognition algorithms and human annotators; the annotations can then be directly modified by the researchers to create their own, improved, annotated datasets. In this paper, we describe the tool's features and implementation that facilitate convenient exploration and annotation of multi-day data and demonstrate its use in generating activity annotations.

Macromolecular modeling and design in Rosetta: recent methods and frameworks.

The Rosetta software for macromolecular modeling, docking and design is extensively used in laboratories worldwide. During two decades of development by a community of laboratories at more than 60 institutions, Rosetta has been continuously refactored and extended. Its advantages are its performance and interoperability between broad modeling capabilities. Here we review tools developed in the last 5 years, including over 80 methods. We discuss improvements to the score function, user interfaces and usability. Rosetta is available at http://www.rosettacommons.org.

Preinjury Narcotic Use Does Not Affect Treatment for Compartment Syndrome.

OBJECTIVES: To determine the association of preinjury opioid use on incidence of fasciotomy after lower extremity trauma. DESIGN: Retrospective case-control study. SETTING: Level 1 trauma center. PATIENTS/PARTICIPANTS: We identified 245 consecutive patients treated with fasciotomy for compartment syndrome of the lower extremity from 2011 to 2016. Of these, 115 were excluded for isolated vascular injury without fracture, age younger than 18 years, out-of-state residence, nontraumatic etiology, and/or incomplete opioid records. Three hundred ninety age- and sex-matched patients with tibial fractures not requiring fasciotomy were selected for comparison. INTERVENTION: Review of demographics, injury characteristics, and opioid prescriptions. MAIN OUTCOME MEASUREMENTS: Rate of preinjury narcotic use. RESULTS: There was no significant difference in chronic opioid use between patients requiring fasciotomy and those who did not (odds ratio = 0.80, 95% confidence interval: 0.43-1.50, P = 0.49). There was no significant difference in average morphine milligram equivalents (MME)/day (66.6 vs. 77.4, P = 0.68). There was no significant difference in active opioid use (odds ratio = 0.76, 95% confidence interval: 0.45-1.29, P = 0.30). There was no significant difference in average MME/day (69.3 vs. 75.6, P = 0.80) for active narcotic users. CONCLUSION: There were no differences in the rate or average MME/day of preinjury opioid use between patients with a tibia fracture treated with or without fasciotomy for compartment syndrome. These results indicate that pre-existing opioid use does not interfere with the accurate diagnosis of compartment syndrome in trauma patients. The diagnosis and treatment of compartment syndrome is not affected by preinjury narcotic use and potential associations with opiate-induced hyperalgesia. LEVEL OF EVIDENCE: Prognostic Level III. See Instructions for Authors for a complete description of levels of evidence.

Expertise and Engagement: Re-Designing Citizen Science Games With Players' Minds in Mind.

Many studies have already shown that games can be a useful tool to make boring or difficult tasks more engaging. However, with serious game design being a relatively nascent field, such experiences can still be hard to learn and not very motivating. In this paper, we explore the use of learning and motivation frameworks to improve player experience in the well-known citizen science game Foldit. Using Cognitive Load Theory (CLT) and Self Determination Theory (SDT), we developed six interface and mechanical changes to the tutorial levels in Foldit designed to increase engagement and retention. We tested these features with new players of Foldit and collected both behavioral data, using game metrics, and prior experience data, using self-report measures. This study offers three major contributions: (1) we document the process of operationalizing CLT and SDT as new game features, a unique methodology not used in game design previously; (2) the user interface, specifically the level selection screen, significantly impacts how players progress through the game; and (3) a player's expertise, whether from prior domain knowledge or prior gaming experience, increases their engagement. We discuss both implications of these findings as well as how these implementations can generalize to other designs.

Building de novo cryo-electron microscopy structures collaboratively with citizen scientists.

With the rapid improvement of cryo-electron microscopy (cryo-EM) resolution, new computational tools are needed to assist and improve upon atomic model building and refinement options. This communication demonstrates that microscopists can now collaborate with the players of the computer game Foldit to generate high-quality de novo structural models. This development could greatly speed the generation of excellent cryo-EM structures when used in addition to current methods.

Designing Videogames to Crowdsource Accelerometer Data Annotation for Activity Recognition Research.

Human activity recognition using wearable accelerometers can enable in-situ detection of physical activities to support novel human-computer interfaces and interventions. However, developing valid algorithms that use accelerometer data to detect everyday activities often requires large amounts of training datasets, precisely labeled with the start and end times of the activities of interest. Acquiring annotated data is challenging and time-consuming. Applied games, such as human computation games (HCGs) have been used to annotate images, sounds, and videos to support advances in machine learning using the collective effort of "non-expert game players." However, their potential to annotate accelerometer data has not been formally explored. In this paper, we present two proof-of-concept, web-based HCGs aimed at enabling game players to annotate accelerometer data. Using results from pilot studies with Amazon Mechanical Turk players, we discuss key challenges, opportunities, and, more generally, the potential of using applied videogames for annotating raw accelerometer data to support activity recognition research.

De novo protein design by citizen scientists.

Online citizen science projects such as GalaxyZoo1, Eyewire2 and Phylo3 have proven very successful for data collection, annotation and processing, but for the most part have harnessed human pattern-recognition skills rather than human creativity. An exception is the game EteRNA4, in which game players learn to build new RNA structures by exploring the discrete two-dimensional space of Watson-Crick base pairing possibilities. Building new proteins, however, is a more challenging task to present in a game, as both the representation and evaluation of a protein structure are intrinsically three-dimensional. We posed the challenge of de novo protein design in the online protein-folding game Foldit5. Players were presented with a fully extended peptide chain and challenged to craft a folded protein structure and an amino acid sequence encoding that structure. After many iterations of player design, analysis of the top-scoring solutions and subsequent game improvement, Foldit players can now-starting from an extended polypeptide chain-generate a diversity of protein structures and sequences that encode them in silico. One hundred forty-six Foldit player designs with sequences unrelated to naturally occurring proteins were encoded in synthetic genes; 56 were found to be expressed and soluble in Escherichia coli, and to adopt stable monomeric folded structures in solution. The diversity of these structures is unprecedented in de novo protein design, representing 20 different folds-including a new fold not observed in natural proteins. High-resolution structures were determined for four of the designs, and are nearly identical to the player models. This work makes explicit the considerable implicit knowledge that contributes to success in de novo protein design, and shows that citizen scientists can discover creative new solutions to outstanding scientific challenges such as the protein design problem.

Creating custom Foldit puzzles for teaching biochemistry.

The computer game Foldit is currently widely used as a biology and biochemistry teaching aid. Herein, we introduce a new feature of Foldit called "custom contests" that allows educators to create puzzles that fit their curriculum. The effectiveness of the custom contests is demonstrated by the use of five distinct custom contests in an upper-level biochemistry class. The new custom contest feature can be implemented in classes ranging from middle school to graduate school to enable educators to best complement their current curriculum. © 2019 International Union of Biochemistry and Molecular Biology, 47(2): 133-139, 2019.

Large-Scale Analysis of Visualization Options in a Citizen Science Game.

Visualization is a valuable tool in problem solving, especially for citizen science games. In this study, we analyze data from 36,351 unique players of the citizen science game Foldit over a period of 5 years to understand how their choice of visualization options are affected by expertise and problem type. We identified clusters of visualization options, and found differences in how experts and novices view puzzles and that experts differentially change their views based on puzzle type. These results can inform new design approaches to help both novice and expert players visualize novel problems, develop expertise, and problem solve.

Inferring and Comparing Game Difficulty Curves using Player-vs-Level Match Data.

Prior work has focused on formalizing difficulty curves by using function composition to give precise definitions to curves and their transformations. However, the proposed framework was demonstrated using a single game, and the curves and transformations were defined with respect to the game's ratings-based dynamic difficulty system. In this work, we infer difficulty curves from gameplay data using a method that is based on the aforementioned difficulty system but that can also be generalized to other games for which information on player-vs-level win/loss outcomes is available. Moreover, since this method uses the same difficulty mechanism as past work, it lets us similarly leverage function composition to compare difficulty curves across games, having either a fixed or dynamic level ordering, using a clearly defined vocabulary. We use four different games to demonstrate our method, which relies on an adjustment to traditional playback of ratings-based match data, which we also present in this work.

Repurposing Citizen Science Games as Software Tools for Professional Scientists.

Scientific software is often developed with professional scientists in mind, resulting in complex tools with a steep learning curve. Citizen science games, however, are designed for citizen scientists- members of the general public. These games maintain scientific accuracy while placing design goals such as usability and enjoyment at the forefront. In this paper, we identify an emerging use of game-based technology, in the repurposing of citizen science games to be software tools for professional scientists in their work. We discuss our experience in two such repurposings: Foldit, a protein folding and design game, and Eyewire, a web-based 3D neuron reconstruction game. Based on this experience, we provide evidence that the software artifacts produced for citizen science can be useful for professional scientists, and provide an overview of key design principles we found to be useful in the process of repurposing.

A Monte Carlo Approach to Skill-Based Automated Playtesting.

In order to create well-crafted learning progressions, designers guide players as they present game skills and give ample time for the player to master those skills. However, analyzing the quality of learning progressions is challenging, especially during the design phase, as content is ever-changing. This research presents the application of Stratabots-automated player simulations based on models of players with varying sets of skills-to the human computation game Foldit. Stratabot performance analysis coupled with player data reveals a relatively smooth learning progression within tutorial levels, yet still shows evidence for improvement. Leveraging existing general gameplaying algorithms such as Monte Carlo Evaluation can reduce the development time of this approach to automated playtesting without losing predicitive power of the player model.

To Three or not to Three: Improving Human Computation Game Onboarding with a Three-Star System.

While many popular casual games use three-star systems, which give players up to three stars based on their performance in a level, this technique has seen limited application in human computation games (HCGs). This gives rise to the question of what impact, if any, a three-star system will have on the behavior of players in HCGs. In this work, we examined the impact of a three-star system implemented in the protein folding HCG Foldit. We compared the basic game's introductory levels with two versions using a three-star system, where players were rewarded with more stars for completing levels in fewer moves. In one version, players could continue playing levels for as many moves as they liked, and in the other, players were forced to reset the level if they used more moves than required to achieve at least one star on the level. We observed that the three-star system encouraged players to use fewer moves, take more time per move, and replay completed levels more often. We did not observe an impact on retention. This indicates that three-star systems may be useful for re-enforcing concepts introduced by HCG levels, or as a flexible means to encourage desired behaviors.

Foldit Standalone: a video game-derived protein structure manipulation interface using Rosetta.

SUMMARY: Foldit Standalone is an interactive graphical interface to the Rosetta molecular modeling package. In contrast to most command-line or batch interactions with Rosetta, Foldit Standalone is designed to allow easy, real-time, direct manipulation of protein structures, while also giving access to the extensive power of Rosetta computations. Derived from the user interface of the scientific discovery game Foldit (itself based on Rosetta), Foldit Standalone has added more advanced features and removed the competitive game elements. Foldit Standalone was built from the ground up with a custom rendering and event engine, configurable visualizations and interactions driven by Rosetta. Foldit Standalone contains, among other features: electron density and contact map visualizations, multiple sequence alignment tools for template-based modeling, rigid body transformation controls, RosettaScripts support and an embedded Lua interpreter. AVAILABILITY AND IMPLEMENTATION: Foldit Standalone is available for download at https://fold.it/standalone , under the Rosetta license, which is free for academic and non-profit users. It is implemented in cross-platform C ++ and binary executables are available for Windows, macOS and Linux. CONTACT: scooper@ccs.neu.edu.

Generative Anatomy Modeling Language (GAML).

BACKGROUND: This paper presents the Generative Anatomy Modeling Language (GAML) for generating variation of 3D virtual human anatomy in real-time. This framework provides a set of operators for modification of a reference base 3D anatomy. The perturbation of the 3D models is satisfied with nonlinear geometry constraints to create an authentic human anatomy. METHODS: GAML was used to create 3D difficult anatomical scenarios for virtual simulation of airway management techniques such as Endotracheal Intubation (ETI) and Cricothyroidotomy (CCT). Difficult scenarios for each technique were defined and the model variations procedurally created with GAML. CONCLUSION: This study presents details of the GAML design, set of operators, types of constraints. Cases of CCT and ETI difficulty were generated and confirmed by expert surgeons. Execution performance pertaining to an increasing complexity of constraints using nonlinear programming was in real-time execution.

Institutional and Seasonal Variations in the Incidence and Causative Organisms for Posttraumatic Infection following Open Fractures.

OBJECTIVES: The current literature focuses on wound severity, time to debridement, and antibiotic administration with respect to risk of infection after open fracture. The purpose of this analysis was to determine if either the incidence of posttraumatic infection or causative organism varies with treating institution or the season in which the open fracture occurred. DESIGN: Retrospective review. SETTING: Seven level 1 regional referral trauma centers located in each of the 7 climatic regions of the continental United States (Northwest, High Plains, Midwest/Ohio Valley, New England/Mid-Atlantic, Southeast, South, and Southwest). PATIENTS/PARTICIPANTS: Five thousand one hundred twenty-seven skeletally mature patients with open extremity fractures treated between 2008 and 2012 at one of the 7 institutions. INTERVENTION: Open reduction and internal fixation of fracture following institutional protocol for antibiotic prophylaxis, debridement, and soft-tissue management. MAIN OUTCOME MEASUREMENTS: Seasonal variation on the incidence of infection and the causative organism after treatment for open fracture as recorded by each individual treating institution. Charts were analyzed to extract information regarding date of injury, Gustilo-Anderson type of open fracture, subsequent treatment for a posttraumatic wound infection, and the causative organisms. Patients were placed into one of the 4 groups based on the time of year that the injury occurred: spring (March-May), summer (June-August), fall (September-November), and winter (December-February). Univariate/multivariate analyses and Fisher test were used to assess whether any observed differences were of statistical significance. RESULTS: The overall incidence of infection for all open fractures across the 7 different institutions was 7.6% and this did not vary significantly by season. There were, however, significant differences in overall infection rates between the different institutions: Southeast 4.3%, Northwest 13%, Northeast 7.7%, Southwest 9.3%, Midwest/Ohio Valley 5.5%, High Plains 14.6%, and South 7.4%. The following institutions demonstrated a significant seasonal variation in the incidence of infection: Northwest = fall 11% versus winter 18.5%, Southwest = winter 1.5% and fall 17.3%, Northeast = winter 5.2% and spring 9.7%, and Southeast = fall 2.8% and spring 6.0%. The High Plains, Midwest/Ohio Valley, and Southern institutions did not demonstrate a significant seasonal variation in infection rates. Finally, the most commonly encountered causative organism varied not only by region, but by season as well. Staphylococcus aureus (both methicillin sensitive and resistant) continues to be the most prevalent organism in the continental United States. CONCLUSIONS: A substantial seasonal and institutional variation exists regarding the incidence of infection and causative organisms for posttraumatic wound infection after open fractures. Although this may represent a difference in treatment regimens between individual surgeons and institutions, a decades-old general nation-wide empiric antibiotic prophylaxis regimen for all open fractures may in fact be outdated and suboptimal. We recommend that surgeons consult with their infectious disease colleagues to better understand the seasonal variation of infection and causative organism for their individual hospital, and adjust their prophylactic and treatment regimens accordingly. LEVEL OF EVIDENCE: Prognostic Level III. See Instructions for Authors for a complete description of levels of evidence.