Digital phenotyping data and anomaly detection methods to assess changes in mood and anxiety symptoms across a transdiagnostic clinical sample.

INTRODUCTION: Clinical assessment of mood and anxiety change often relies on clinical assessment or self-reported scales. Using smartphone digital phenotyping data and resulting markers of behavior (e.g., sleep) to augment clinical symptom scores offers a scalable and potentially more valid method to understand changes in patients' state. This paper explores the potential of using a combination of active and passive sensors in the context of smartphone-based digital phenotyping to assess mood and anxiety changes in two distinct cohorts of patients to assess the preliminary reliability and validity of this digital phenotyping method. METHODS: Participants from two different cohorts, each n = 76, one with diagnoses of depression/anxiety and the other schizophrenia, utilized mindLAMP to collect active data (e.g., surveys on mood/anxiety), along with passive data consisting of smartphone digital phenotyping data (geolocation, accelerometer, and screen state) for at least 1 month. Using anomaly detection algorithms, we assessed if statistical anomalies in the combination of active and passive data could predict changes in mood/anxiety scores as measured via smartphone surveys. RESULTS: The anomaly detection model was reliably able to predict symptom change of 4 points or greater for depression as measured by the PHQ-9 and anxiety as measured for the GAD-8 for both patient populations, with an area under the ROC curve of 0.65 and 0.80 for each respectively. For both PHQ-9 and GAD-7, these AUCs were maintained when predicting significant symptom change at least 7 days in advance. Active data alone predicted around 52% and 75% of the symptom variability for the depression/anxiety and schizophrenia populations respectively. CONCLUSION: These results indicate the feasibility of anomaly detection for predicting symptom change in transdiagnostic cohorts. These results across different patient groups, different countries, and different sites (India and the US) suggest anomaly detection of smartphone digital phenotyping data may offer a reliable and valid approach to predicting symptom change. Future work should emphasize prospective application of these statistical methods.

Spatial attention across perception and action.

We hypothesize that a shared spatial attention mechanism is used for both perception and action. To this end we created a new dual-task version of the classical Simon task. In one task, the spatial-input task, associated with input spatial attention, participants named one shape out of two bilaterally presented colored shapes. In a second task, the spatial-output task, associated with output spatial attention, participants discriminated between high and low pitch tones by pressing either a left or a right key. In Experiment 1, input for both tasks appeared simultaneously, and participants were instructed not to prioritize either task. A between tasks Simon-like effect was found for responses to both tasks. Reaction times were shorter when the side of the relevant shape in the spatial-input task and the side of the correct response in the spatial-output task were congruent. In Experiment 2, we manipulated the stimulus-onset asynchrony (SOA) between the inputs for the two tasks and showed that the Simon-like effect remained intact at all SOAs. Experiment 3 was similar to Experiment 1 except that the vocal response for the spatial-input task was not speeded. A Simon-like effect was still observed. Experiment 4 was the same as Experiment 3 except that the non-speeded response for the spatial-input task was manual rather than vocal. No Simon-like effect was observed in this experiment. Our results support a shared spatial attention mechanism involved in the Simon effect and indicate that this spatial attention mechanism is shared by perception and action.

Digital phenotyping correlates of mobile cognitive measures in schizophrenia: A multisite global mental health feasibility trial.

Traditional cognitive assessments in schizophrenia are time-consuming and necessitate specialized training, making routine evaluation challenging. To overcome these limitations, this study investigates the feasibility and advantages of utilizing smartphone-based assessments to capture both cognitive functioning and digital phenotyping data and compare these results to gold standard measures. We conducted a secondary analysis of data from 76 individuals with schizophrenia, who were recruited across three sites (one in Boston, two in India) was conducted. The open-source mindLAMP smartphone app captured digital phenotyping data and Trails A/B assessments of attention / memory for up to 12 months. The smartphone-cognitive tasks exhibited potential for normal distribution and these scores showed small but significant correlations with the results from the Brief Assessment of Cognition in Schizophrenia, especially the digital span and symbol coding tasks (r2 = 0.21). A small but significant correlation (r2 = 0.29) between smartphone-derived cognitive scores and health-related behaviors such as sleep duration patterns was observed. Smartphone-based cognitive assessments show promise as cross-cultural tools that can capture relevant data on momentary states among individuals with schizophrenia. Cognitive results related to sleep suggest functional applications to digital phenotyping data, and the potential of this multimodal data approach in research.

Relapse prediction in schizophrenia with smartphone digital phenotyping during COVID-19: a prospective, three-site, two-country, longitudinal study.

Smartphone technology provides us with a more convenient and less intrusive method of detecting changes in behavior and symptoms that typically precede schizophrenia relapse. To take advantage of the aforementioned, this study examines the feasibility of predicting schizophrenia relapse by identifying statistically significant anomalies in patient data gathered through mindLAMP, an open-source smartphone app. Participants, recruited in Boston, MA in the United States, and Bangalore and Bhopal in India, were invited to use mindLAMP for up to a year. The passive data (geolocation, accelerometer, and screen state), active data (surveys), and data quality metrics collected by the app were then retroactively fed into a relapse prediction model that utilizes anomaly detection. Overall, anomalies were 2.12 times more frequent in the month preceding a relapse and 2.78 times more frequent in the month preceding and following a relapse compared to intervals without relapses. The anomaly detection model incorporating passive data proved a better predictor of relapse than a naive model utilizing only survey data. These results demonstrate that relapse prediction models utilizing patient data gathered by a smartphone app can warn the clinician and patient of a potential schizophrenia relapse.

Assessment of Mental Health Services Available Through Smartphone Apps.

Importance: As more patients and clinicians are turning to mental health smartphone apps to expand access to services, little is known about the current state of the app marketplaces and what these apps are actually offering in terms of features, privacy, price, and services. Objective: To assess the current state of mental health apps, explore the association between app privacy scores and popularity as measured by star ratings and downloads, and to understand opportunities and challenges facing the commercial app landscape. Design, Setting, and Participants: This cross-sectional study had trained raters using the public-facing M-Health Index and Navigation Database (MIND) to assess and review 578 mental health apps. The sample of apps used in this analysis were pulled from MIND and include apps across various conditions including schizophrenia, eating disorders, sleep, and more. Analysis of these apps was conducted in June 2022. Exposures: There were 578 mental health apps rated across 105 dimensions derived from the American Psychiatric Association's app evaluation framework. Main Outcomes and Measures: App raters assessed each app across 6 categories: (1) app origin and accessibility, (2) privacy and security, (3) clinical foundation, (4) features and engagement, (5) inputs and outputs, and (6) interoperability. Privacy scores were determined by 5 MIND criteria, including (1) having a privacy policy, (2) reporting security measures in place, (3) declaring data use and purpose, (4) allowing for the deletion of data, and (5) allowing users to opt out of data collection. Correlations between privacy scores and popularity metrics (star ratings and number of downloads) were measured. Results: This study included 578 mental health apps that were identified, assessed, and analyzed across 105 MIND dimensions. Psychoeducation, goal setting, and mindfulness were among the top app features. Of the 578 apps analyzed, 443 (77%) had a privacy policy. This analysis of apps with a privacy policy revealed that there was no statistically significant correlation between privacy scores and Apple App Store (r = 0.058, P = .29) or Google Play Store star ratings (r = 0.041; P = .48). The number of app downloads on the Google Play Store, however, was weakly correlated with privacy scores (χ25 = 22.1; P < .001). Conclusions and Relevance: In this cross-sectional study of mental health apps, findings indicate that the current app marketplaces primarily offered basic features such as psychoeducation, goal tracking, and mindfulness but fewer innovative features such as biofeedback or specialized therapies. Privacy challenges remained common, and app popularity metrics provided little help in identifying apps with more privacy.

Involuntary strategy-dependent dual task performance.

Previous studies showed that some dual tasks can be performed simultaneously without costs. Yet, a variable SOA between the inputs to such tasks leads to strategic, often involuntary, prioritization of one of the two tasks. Here we explore the boundary conditions for this involuntary or exogenous strategy. In Experiment 1, subjects were initially trained on dual task performance where the input to the two tasks is presented simultaneously (0 SOA). We used two tasks that under such conditions can be performed without costs and indeed subjects displayed perfect sharing of the tasks. Subjects then performed the same two tasks but with a variable SOA (0, 50, 150, 800 ms). This manipulation led to a serial-like performance of the two tasks even in trials with 0 SOA. In Experiment 2, subjects participated in eight sessions. Within each session, they performed in alternation blocks with a fixed 0 SOA and blocks with a variable SOA. Subjects displayed perfect sharing in the pure 0 SOA blocks but performed the two tasks serially in the mixed SOA blocks despite receiving identical instructions. These findings demonstrate that task context is a powerful factor in dual task performance and may lead subjects to involuntarily exhibit dual task costs even in conditions where they can perform the tasks without any costs. Moreover, these findings strongly suggest that costs observed in PRP studies reflect the use of such exogenous strategies rather than a general structural dual task limitation.

What is it that color determinants determine? The relation between the rorschach inkblot method and cognitive object-recognition processes.

We sought to demonstrate a relation between the Determinants in the Rorschach Inkblot Method (Rorschach, 1921) and fundamental properties of the participant's cognitive (visual) system by examining whether the report about Color Determinants is related to basic cognitive processes concerned with color of visual objects. In Experiment 1, we established an object-naming task that is sensitive to the objects' color. Participants were strongly influenced by the object's color, responding fastest when objects appeared in their typical color and slowest when the object's color was atypical. In Experiment 2, we examined the relationship between the Color Determinants in the Rorschach Inkblot Method and the magnitude of the color effect in the object-naming task of Experiment 1. It was found that the effect of color in the object-naming task was correlated with the type of color responses in the Rorschach Inkblot Method. The results support an "early" cognitive account of the Determinants. We discuss implications concerning the theory of the Rorschach and the relation between emotion, personality, and cognition.

Lack of Automatic Vocal Response Learning While Reading Aloud.

Research on implicit sequence learning with the Serial Reaction Task (SRT) has demonstrated that people automatically acquire knowledge about fixed repeating sequences of responses and can transfer response sequence knowledge to novel stimuli. Such demonstrations are, however, mostly limited to setups with visual stimuli and manual responses. Here we systematically follow up on scarce attempts to demonstrate implicit sequence learning in word reading. While the literature on implicit sequence learning can be taken to suggest that sequence knowledge is acquired and affecting performance in word reading, we show that neither is the case in a series of four experiments. Sequence knowledge was acquired and affecting performance in color naming but not in word reading. On the one hand, we observed slowing of voice-onset times in off-sequence as compared to regularly sequenced trials when people named the color of a centrally presented disk. Yet, hardly any effect was observed when the very same sequence of words was verbalized in word reading instead. Transfer of sequence knowledge to and from color naming was not observed, either. This contrasts with sequence learning studies with manual responses, which have been taken to suggest that a fixed and repeating sequence of responses is sufficient for learning to occur even in fast choice reaction tasks and to transfer across stimuli as long as the sequence of responses remains intact. Rather, in line with dimensional action accounts of task performance, the results underline the role of translation between processing streams for implicit sequence learning.

The effect of pre-cueing on spatial attention across perception and action.

It is well established that processes of perception and action interact. A key question concerns the role of attention in the interaction between perception-action processes. We tested the hypothesis that spatial attention is shared by perception and action. We created a dual-task paradigm: In one task, spatial information is relevant for perception (spatial-input task) but not for action, and in a second task, spatial information is relevant for action (spatial-output task) but not for perception. We used endogenous pre-cueing, with two between-subjects conditions: In one condition the cue was predictive only for the target location in the spatial-input task; in a second condition the cue was predictive only for the location of the response in the spatial-output task. In both conditions, the cueing equally affected both tasks, regardless of the information conveyed by the cue. This finding directly supports the shared input-output attention hypothesis.

The effect of novel distractors on performance in focused attention tasks: a cognitive-psychophysiological approach.

In the present study, we examined whether or not novel stimuli affect performance in a focused attention task. Participants responded to a central target while an irrelevant distractor in the visual display was occasionally changed. In Experiment 1, both target and distractor were presented centrally within the focus of attention. In Experiment 2, a central target was presented along with an irrelevant distractor at a peripheral location, outside the focus of attention. Novel distractors were associated with longer latencies and enhanced orienting responses (as measured by skin conductance responses) only when presented at an attended location. In contrast, as is demonstrated in Experiment 3, the same peripheral novel distractors interfered with task performance when they possessed task-relevant information. These results indicate that there is a fundamental difference between novel stimuli and task-relevant stimuli. Whereas the former exert influence only within the focus of attention, the latter affect performance even when positioned in an unattended location. Our findings have important implications for the operation of visual attention.

Location specificity in response selection processes for visual stimuli.

We used the psychological refractory period paradigm, in which participants respond to two successive tasks (T1 and T2). We created in T2 spatial and color Simon effects, known to be caused by response selection processes. Previous studies in which the spatial Simon effect was manipulated in T2 showed that this effect was underadditive, with stimulus onset asynchrony (SOA) between the targets for T1 and T2. In Experiment 1, we replicated these results with two versions of the spatial Simon effect. In contrast, in Experiment 2 we manipulated two versions of a color Simon effect, revealing an additive relation between the color Simon effect and SOA. These results suggest that the underadditivity obtained with the spatial Simon effect is due to its spatial nature, and that space may play a unique role in response selection processes.

Behavioral and physiological measures in the detection of concealed information.

The authors examined the incremental validity of the reaction time (RT) measure beyond that of skin conductance response (SCR) in the detection of concealed information. Participants performed a Stroop-like task in which they named the color of critical and neutral words. Results show that the SCR highly differentiated between the relevant and neutral words. However, the RT demonstrated a significant differentiation only when the critical words denoted personally significant items (e.g., one's own name) and not when they denoted crime-relevant items related to a simulated crime. In both cases, combining the 2 measures yielded no advantage over the use of SCR alone. Thus, although behavioral measures may differentiate between relevant and neutral information in some cases, their practical use is questionable.

Dissociations of personally significant and task-relevant distractors inside and outside the focus of attention: a combined behavioral and psychophysiological study.

Studies of attentional capture by personally significant stimuli have reached inconsistent results, possibly because of improper control of the participants' attention. In the present study, the authors controlled visual attention by using a Stroop-like task. Participants responded to a central color and ignored a word presented either centrally (i.e., at the focus of attention) or peripherally (i.e., outside the focus of attention). Central words led to slower reaction times and larger orienting responses for significant items than for neutral items. These effects largely disappeared when the words appeared in a peripheral location. The peripheral words interfered with performance when they were relevant to task demands. These results indicate that there is a fundamental difference between task-relevant words and personally significant words: The former capture attention even when presented peripherally, whereas the latter do not.

Bimanual interference associated with the selection of target locations.

Four experiments were conducted to identify the locus of interference observed during the preparation of bimanual reaching movements. Target locations were specified by color, and the right-hand and left-hand targets could be either the same or a different color. Movements of different amplitudes (Experiment 1) or different directions (Experiment 2) to targets of the same color were initiated more quickly than symmetric movements to targets of different colors. These results indicate that costs observed during bimanual movements arise during target selection rather than during motor programming. Experiments 3 and 4 further examined the interference associated with target selection. Reaction time costs were found with unimanual movements when the target was presented among distractors associated with responses for the other hand. Interference observed during bimanual reaching appears to reflect difficulty in segregating the response rules assigned to each hand.

The role of cross-modal associations in statistical learning.

Our environment is richly structured, with objects producing correlated information within and across sensory modalities. A prominent challenge faced by our perceptual system is to learn such regularities. Here, we examined statistical learning and addressed learners' ability to track transitional probabilities between elements in the auditory and visual modalities. Specifically, we investigated whether cross-modal information affects statistical learning within a single modality. Participants were familiarized with a statistically structured modality (e.g., either audition or vision) accompanied by different types of cues in a second modality (e.g., vision or audition). The results revealed that statistical learning within either modality is affected by cross-modal information, with learning being enhanced or reduced according to the type of cue provided in the second modality.

Relevance-based selectivity: the case of implicit learning.

Learning the structure of the environment (e.g., what usually follows what) enables animals to behave in an effective manner and prepare for future events. Unintentional learning is capable of efficiently producing such knowledge as has been demonstrated with the Artificial Grammar Learning paradigm (AGL), among others. It has been argued that selective attention is a necessary and sufficient condition for visual implicit learning. Experiment 1 shows that spatial attention is not sufficient for implicit learning. Learning does not occur if the stimuli instantiating the structure are task irrelevant. In a second experiment, we demonstrate that this holds even with abundance of available attentional resources. Together, these results challenge the current view of the relations between attention, resources, and implicit learning.

Implicit sequence learning based on instructed task set.

How does the way we code and control actions influence automatic skill acquisition processes? Wenke and Frensch (2005) showed that instructions can lead participants to code spatial responses based on color. Here, we tested in 3 experiments to what extent response labeling and instruction-based response coding can determine what is learned in implicit sequence learning. Instructions mapped 4 gray shape stimuli to 1 of the 4 keys each in a serial reaction task, referring to the keys in terms of either their color or their spatial location. In Experiments 1 and 2 we found that people in the color instruction conditions used color for action control and acquired sequence knowledge containing color: They were susceptible to irrelevant stimulus colors at transfer and could transfer color sequence knowledge to a new arrangement of response positions and fingers, whereas participants who had received spatial instructions could not. Implicit sequence learning was thus surprisingly flexible. Depending on whether an arbitrary nonspatial response feature was used or not used to explain the stimulus-response mappings, we either found or did not find evidence that this feature became part of action control and sequence learning. Furthermore, Experiment 3 suggested that response position might become part of the sequence knowledge even if instructions do not emphasize this response feature. Together, the findings suggest that implicit sequence learning is based on action control, which in turn strongly, but not entirely, depends on which response features are used to explain the stimulus-response mappings in the instructions.

The role of Gestalt grouping principles in visual statistical learning.

A major issue in visual scene recognition involves the extraction of recurring chunks from a sequence of complex scenes. Previous studies have suggested that this kind of learning is accomplished according to Bayesian principles that constrain the types of extracted chunks. Here we show that perceptual grouping cues are also incorporated in this Bayesian model, providing additional evidence for the possible span of chunks. Experiment 1 replicates previous results showing that observers can learn three-element chunks without learning smaller, two-element chunks embedded within them. Experiment 2 shows that the very same embedded chunks are learned if they are grouped by perceptual cues, suggesting that perceptual grouping cues play an important role in chunk extraction from complex scenes.

Differential dynamics of spatial and non-spatial stimulus-response compatibility effects: a dual task LRP study.

Choice reaction times are shorter when stimulus and response features are compatible rather than incompatible. Recent studies revealed that spatial compatibility effects in Simon tasks are strongly attenuated when there is temporal overlap with a different high-priority task. In contrast, non-spatial variants of the Simon task appear to be unaffected by task overlap. The present study used the lateralized readiness potential (LRP) within a dual task design to elucidate the dynamics underlying these differential effects for a color and a spatial variant of the Simon task. In the color version there was no sign of early response priming by irrelevant stimulus features in the LRP. The color compatibility effect was independent of task overlap and reflected in the LRP onset latency. In contrast, in the spatial version, priming by irrelevant stimulus location showed up and was mirrored by early LRP activation. Response priming and the corresponding Simon effect, however, were present only in case of little temporal overlap with the primary task. The absence of spatial compatibility effects at strong temporal overlap suggests that response conflicts due to stimulus-related priming depend on the availability of processing resources.

Personally-significant information affects performance only within the focus of attention: a direct manipulation of attention.

It has been suggested that personally significant (PS) information interferes with performance only when presented within the focus of attention. However, this claim was never tested by a systematic manipulation of attention, but only by using correlative measures of its locus. We addressed this issue in two experiments, utilizing a cued visual search paradigm that allowed us to directly manipulate attention and to measure behavioral and physiological responses. One of the stimuli in the search display had a higher luminance value (i.e., was cued), and, orthogonally, one of the stimuli could be a PS or neutral name. When the cue did not predict target location, PS distractors mildly interfered with task performance, regardless of the cue's location. However, when the cue predicted target location, responses were facilitated for cued targets, indicating that attention was shifted to the cue. Importantly, PS distractors interfered with task performance and elicited enhanced orienting responses only when they were cued. This implies that PS information affects performance only when presented within but not outside the focus of attention.