Measuring the Reliability of a Gamified Stroop Task: Quantitative Experiment.

BACKGROUND: Few gamified cognitive tasks are subjected to rigorous examination of psychometric properties, despite their use in experimental and clinical settings. Even small manipulations to cognitive tasks require extensive research to understand their effects. OBJECTIVE: This study aims to investigate how game elements can affect the reliability of scores on a Stroop task. We specifically investigated performance consistency within and across sessions. METHODS: We created 2 versions of the Stroop task, with and without game elements, and then tested each task with participants at 2 time points. The gamified task used points and feedback as game elements. In this paper, we report on the reliability of the gamified Stroop task in terms of internal consistency and test-retest reliability, compared with the control task. We used a permutation approach to evaluate internal consistency. For test-retest reliability, we calculated the Pearson correlation and intraclass correlation coefficients between each time point. We also descriptively compared the reliability of scores on a trial-by-trial basis, considering the different trial types. RESULTS: At the first time point, the Stroop effect was reduced in the game condition, indicating an increase in performance. Participants in the game condition had faster reaction times (P=.005) and lower error rates (P=.04) than those in the basic task condition. Furthermore, the game condition led to higher measures of internal consistency at both time points for reaction times and error rates, which indicates a more consistent response pattern. For reaction time in the basic task condition, at time 1, rSpearman-Brown=0.78, 95% CI 0.64-0.89. At time 2, rSpearman-Brown=0.64, 95% CI 0.40-0.81. For reaction time, in the game condition, at time 1, rSpearman-Brown=0.83, 95% CI 0.71-0.91. At time 2, rSpearman-Brown=0.76, 95% CI 0.60-0.88. Similarly, for error rates in the basic task condition, at time 1, rSpearman-Brown=0.76, 95% CI 0.62-0.87. At time 2, rSpearman-Brown=0.74, 95% CI 0.58-0.86. For error rates in the game condition, at time 1, rSpearman-Brown=0.76, 95% CI 0.62-0.87. At time 2, rSpearman-Brown=0.74, 95% CI 0.58-0.86. Test-retest reliability analysis revealed a distinctive performance pattern depending on the trial type, which may be reflective of motivational differences between task versions. In short, especially in the incongruent trials where cognitive conflict occurs, performance in the game condition reaches peak consistency after 100 trials, whereas performance consistency drops after 50 trials for the basic version and only catches up to the game after 250 trials. CONCLUSIONS: Even subtle gamification can impact task performance albeit not only in terms of a direct difference in performance between conditions. People playing the game reach peak performance sooner, and their performance is more consistent within and across sessions. We advocate for a closer examination of the impact of game elements on performance.

The Development of Explicit and Implicit Game-Based Digital Behavioral Markers for the Assessment of Social Anxiety.

Social relationships are essential for humans; neglecting our social needs can reduce wellbeing or even lead to the development of more severe issues such as depression or substance dependency. Although essential, some individuals face major challenges in forming and maintaining social relationships due to the experience of social anxiety. The burden of social anxiety can be reduced through accessible assessment that leads to treatment. However, socially anxious individuals who seek help face many barriers stemming from geography, fear, or disparities in access to systems of care. But recent research suggested digital behavioral markers as a way to deliver cheap and easily accessible digital assessment for social anxiety: As earlier work shows, players with social anxiety show similar behaviors in virtual worlds as in the physical world, including tending to walk farther around other avatars and standing farther away from other avatars. The characteristics of the movement behavior in-game can be harnessed for the development of digital behavioral markers for the assessment of social anxiety. In this paper, we investigate whether implicit as well as explicit digital behavioral markers, proposed by prior work, for social anxiety can be used for predicting the level of social anxiety. We show that both, explicit and implicit digital behavioral markers can be harnessed for the assessment. Our findings provide further insights about how game-based digital behavioral markers can be used for the assessment of social anxiety.

Effective Gamification of the Stop-Signal Task: Two Controlled Laboratory Experiments.

BACKGROUND: A lack of ability to inhibit prepotent responses, or more generally a lack of impulse control, is associated with several disorders such as attention-deficit/hyperactivity disorder and schizophrenia as well as general damage to the prefrontal cortex. A stop-signal task (SST) is a reliable and established measure of response inhibition. However, using the SST as an objective assessment in diagnostic or research-focused settings places significant stress on participants as the task itself requires concentration and cognitive effort and is not particularly engaging. This can lead to decreased motivation to follow task instructions and poor data quality, which can affect assessment efficacy and might increase drop-out rates. Gamification-the application of game-based elements in nongame settings-has shown to improve engaged attention to a cognitive task, thus increasing participant motivation and data quality. OBJECTIVE: This study aims to design a gamified SST that improves participants' engagement and validate this gamified SST against a standard SST. METHODS: We described the design of our gamified SST and reported on 2 separate studies that aim to validate the gamified SST relative to a standard SST. In study 1, a within-subject design was used to compare the performance of the SST and a stop-signal game (SSG). In study 2, we added eye tracking to the procedure to determine if overt attention was affected and aimed to replicate the findings from study 1 in a between-subjects design. Furthermore, in both studies, flow and motivational experiences were measured. RESULTS: In contrast, the behavioral performance was comparable between the tasks (P<.87; BF01=2.87), and the experience of flow and intrinsic motivation were rated higher in the SSG group, although this difference was not significant. CONCLUSIONS: Overall, our findings provide evidence that the gamification of SST is possible and that the SSG is enjoyed more. Thus, when participant engagement is critical, we recommend using the SSG instead of the SST.

The Potential of Game-Based Digital Biomarkers for Modeling Mental Health.

BACKGROUND: Assessment for mental health is performed by experts using interview techniques, questionnaires, and test batteries and following standardized manuals; however, there would be myriad benefits if behavioral correlates could predict mental health and be used for population screening or prevalence estimations. A variety of digital sources of data (eg, online search data and social media posts) have been previously proposed as candidates for digital biomarkers in the context of mental health. Playing games on computers, gaming consoles, or mobile devices (ie, digital gaming) has become a leading leisure activity of choice and yields rich data from a variety of sources. OBJECTIVE: In this paper, we argue that game-based data from commercial off-the-shelf games have the potential to be used as a digital biomarker to assess and model mental health and health decline. Although there is great potential in games developed specifically for mental health assessment (eg, Sea Hero Quest), we focus on data gathered "in-the-wild" from playing commercial off-the-shelf games designed primarily for entertainment. METHODS: We argue that the activity traces left behind by natural interactions with digital games can be modeled using computational approaches for big data. To support our argument, we present an investigation of existing data sources, a categorization of observable traits from game data, and examples of potentially useful game-based digital biomarkers derived from activity traces. RESULTS: Our investigation reveals different types of data that are generated from play and the sources from which these data can be accessed. Based on these insights, we describe five categories of digital biomarkers that can be derived from game-based data, including behavior, cognitive performance, motor performance, social behavior, and affect. For each type of biomarker, we describe the data type, the game-based sources from which it can be derived, its importance for mental health modeling, and any existing statistical associations with mental health that have been demonstrated in prior work. We end with a discussion on the limitations and potential of data from commercial off-the-shelf games for use as a digital biomarker of mental health. CONCLUSIONS: When people play commercial digital games, they produce significant volumes of high-resolution data that are not only related to play frequency, but also include performance data reflecting low-level cognitive and motor processing; text-based data that are indicative of the affective state; social data that reveal networks of relationships; content choice data that imply preferred genres; and contextual data that divulge where, when, and with whom the players are playing. These data provide a source for digital biomarkers that may indicate mental health. Produced by engaged human behavior, game data have the potential to be leveraged for population screening or prevalence estimations, leading to at-scale, nonintrusive assessment of mental health.

Improving the Efficacy of Cognitive Training for Digital Mental Health Interventions Through Avatar Customization: Crowdsourced Quasi-Experimental Study.

BACKGROUND: The success of internet-based mental health interventions in practice-that is, in the wild-depends on the uptake and retention of the application and the user's focused attention in the moment of use. Incorporating game-based motivational design into digital interventions delivered in the wild has been shown to increase uptake and retention in internet-based training; however, there are outstanding questions about the potential of game-based motivational strategies to increase engagement with a task in the moment of use and the effect on intervention efficacy. OBJECTIVE: Designers of internet-based interventions need to know whether game-based motivational design strategies can increase in-the-moment engagement and thus improve digital interventions. The aim of this study was to investigate the effects of 1 motivational design strategy (avatar customization) in an example mental health intervention (computerized cognitive training for attention bias modification). METHODS: We assigned 317 participants to either a customized avatar or an assigned avatar condition. After measuring state anxiety (State-Trait Anxiety Inventory), we randomly assigned half of the participants in each condition to either an attentional retraining condition (Attention Bias Modification Training) or a control condition. After training, participants were exposed to a negative mood induction using images with strong negative valance (International Affective Picture System), after which we measured state anxiety again. RESULTS: Avatar customization decreased posttraining state anxiety when controlling for baseline state anxiety for those in the attentional retraining condition; however, those who did not train experienced decreased resilience to the negative mood induction (F1,252=6.86, P=.009, ηp2=.027). This interaction effect suggests that customization increased task engagement with the intervention in the moment of use. Avatar customization also increased avatar identification (F5,252=12.46, P<.001, R2=.23), regardless of condition (F1,252=.79, P=.38). Avatar identification reduced anxiety after the negative mood induction for participants who underwent training but increased poststimulus anxiety for participants who did not undergo training, further suggesting that customization increases engagement in the task (F1,252=6.19, P=.01). The beneficial effect of avatar customization on training was driven by participants who were low in their basic satisfaction of relatedness (F10,248=18.5, P<.001, R2=.43), which is important because these are the participants who are most likely in need of digital interventions for mental health. CONCLUSIONS: Our results suggest that applying motivational design-specifically avatar customization-is a viable strategy to increase engagement and subsequently training efficacy in a computerized cognitive task.

Toward Game-Based Digital Mental Health Interventions: Player Habits and Preferences.

BACKGROUND: Designers of digital interventions for mental health often leverage interactions from games because the intrinsic motivation that results from game-based interventions may increase participation and translate into improved treatment efficacy. However, there are outstanding questions about the suitability (eg, are desktop or mobile interventions more appropriate?) and intervention potential (eg, do people with depression activate enough to play?) of games for mental health. OBJECTIVE: In this paper, we aimed to describe the presently unknown relationship between gaming activity and indicators of well-being so that designers make informed choices when designing game-based interventions for mental health. METHODS: We gathered validated scales of well-being (Beck's Depression Inventory [BDI-II], Patient Health Questionnaire [PHQ-9], trait anxiety [TA], and basic psychological needs satisfaction [BPNS]), play importance (control over game behavior: control; gamer identity: identity), and play behavior (play frequency, platform preferences, and genre preferences) in a Web-based survey (N=491). RESULTS: The majority of our participants played games a few times a week (45.3%, 222/490) or daily (34.3%, 168/490). In terms of depression, play frequency was associated with PHQ-9 (P=.003); PHQ-9 scores were higher for those who played daily than for those who played a few times a week or less. Similarly, for BDI-II (P=.01), scores were higher for those who played daily than for those who played once a week or less. Genre preferences were not associated with PHQ-9 (P=.32) or BDI-II (P=.68); however, platform preference (ie, mobile, desktop, or console) was associated with PHQ-9 (P=.04); desktop-only players had higher PHQ-9 scores than those who used all platforms. Platform preference was not associated with BDI-II (P=.18). In terms of anxiety, TA was not associated with frequency (P=.23), platform preference (P=.07), or genre preference (P=.99). In terms of needs satisfaction, BPNS was not associated with frequency (P=.25) or genre preference (P=.53), but it was associated with platform preference (P=.01); desktop-only players had lower needs satisfaction than those who used all platforms. As expected, play frequency was associated with identity (P<.001) and control (P<.001); those who played more had identified more as a gamer and had less control over their gameplay. Genre preference was associated with identity (P<.001) and control (P<.001); those who played most common genres had higher control over their play and identified most as gamers. Platform preference was not associated with control (P=.80), but was with identity (P=.001); those who played on all devices identified more as a gamer than those who played on mobiles or consoles only. CONCLUSIONS: Our results suggest that games are a suitable approach for mental health interventions as they are played broadly by people across a range of indicators of mental health. We further unpack the platform preferences and genre preferences of players with varying levels of well-being.