Predicting cognitive scores from wearable-based digital physiological features using machine learning: data from a clinical trial in mild cognitive impairment.

BACKGROUND: Continuous assessment and remote monitoring of cognitive function in individuals with mild cognitive impairment (MCI) enables tracking therapeutic effects and modifying treatment to achieve better clinical outcomes. While standardized neuropsychological tests are inconvenient for this purpose, wearable sensor technology collecting physiological and behavioral data looks promising to provide proxy measures of cognitive function. The objective of this study was to evaluate the predictive ability of digital physiological features, based on sensor data from wrist-worn wearables, in determining neuropsychological test scores in individuals with MCI. METHODS: We used the dataset collected from a 10-week single-arm clinical trial in older adults (50-70 years old) diagnosed with amnestic MCI (N = 30) who received a digitally delivered multidomain therapeutic intervention. Cognitive performance was assessed before and after the intervention using the Neuropsychological Test Battery (NTB) from which composite scores were calculated (executive function, processing speed, immediate memory, delayed memory and global cognition). The Empatica E4, a wrist-wearable medical-grade device, was used to collect physiological data including blood volume pulse, electrodermal activity, and skin temperature. We processed sensors' data and extracted a range of physiological features. We used interpolated NTB scores for 10-day intervals to test predictability of scores over short periods and to leverage the maximum of wearable data available. In addition, we used individually centered data which represents deviations from personal baselines. Supervised machine learning was used to train models predicting NTB scores from digital physiological features and demographics. Performance was evaluated using "leave-one-subject-out" and "leave-one-interval-out" cross-validation. RESULTS: The final sample included 96 aggregated data intervals from 17 individuals. In total, 106 digital physiological features were extracted. We found that physiological features, especially measures of heart rate variability, correlated most strongly to the executive function compared to other cognitive composites. The model predicted the actual executive function scores with correlation r = 0.69 and intra-individual changes in executive function scores with r = 0.61. CONCLUSIONS: Our findings demonstrated that wearable-based physiological measures, primarily HRV, have potential to be used for the continuous assessments of cognitive function in individuals with MCI.

Learning in the Target Prevalence Effect.

Rare or low prevalence targets are detected less well than counterparts that occur with higher probability. It stands to reason, though, that before such a deficit is apparent, information about a given target's probability of occurrence must be apprehended. In this study, we investigated how much target experience is necessary for target probabilities to be fully acquired and established within mental task representations. A central finding was that different target probability values required approximately the same amount of target sampling to learn. This was true whether learning about target probabilities from a naive start-point (Experiment 1) or when recalibrating from one probability value to another (Experiment 2). We discuss these findings in relation to how mental task representations are modified when new task-relevant information is received and the attentional consequences of such changes.

Probabilistic and rich individual working memories revealed by a betting game.

When asked to remember a color, do people remember a point estimate (e.g., a particular shade of red), a point estimate plus an uncertainty estimate, or are memory representations rich probabilistic distributions over feature space? We asked participants to report the color of a circle held in working memory. Rather than collecting a single report per trial, we had participants place multiple bets to create trialwise uncertainty distributions. Bet dispersion correlated with performance, indicating that internal uncertainty guided bet placement. While the first bet was on average the most precisely placed, the later bets systematically shifted the distribution closer to the target, resulting in asymmetrical distributions about the first bet. This resulted in memory performance improvements when averaging across bets, and overall suggests that memory representations contain more information than can be conveyed by a single response. The later bets contained target information even when the first response would generally be classified as a guess or report of an incorrect item, suggesting that such failures are not all-or-none. This paradigm provides multiple pieces of evidence that memory representations are rich and probabilistic. Crucially, standard discrete response paradigms underestimate the amount of information in memory representations.

Perception is rich and probabilistic.

When we see a stimulus, e.g. a star-shaped object, our intuition is that we should perceive a single, coherent percept (even if it is inaccurate). But the neural processes that support perception are complex and probabilistic. Simple lines cause orientation-selective neurons across a population to fire in a probabilistic-like manner. Does probabilistic neural firing lead to non-probabilistic perception, or are the representations behind perception richer and more complex than intuition would suggest? To test this, we briefly presented a complex shape and had participants report the correct shape from a set of options. Rather than reporting a single value, we used a paradigm designed to encourage to directly report a representation over shape space-participants placed a series of Gaussian bets. We found that participants could report more than point-estimates of shape. The spread of responses was correlated with accuracy, suggesting that participants can convey a notion of relative imprecision. Critically, as participants placed more bets, the mean of responses show increased precision. The later bets were systematically biased towards the target rather than haphazardly placed around bet 1. These findings strongly indicate that participants were aware of more than just a point-estimate; Perceptual representations are rich and likely probabilistic.

How do expectations change behavior? Investigating the contributions at encoding versus decision-making.

Expectations about the environment play a large role in shaping behavior, but how does this occur? Do expectations change the way we perceive the world, or just our decisions based on unbiased perceptions? We investigated the relative contributions of priors to these 2 stages by manipulating when information about expected color was provided. We compared cases where the prior could affect encoding into perceptual/working memory representations (e.g., when provided prestimulus) against cases where it could not (e.g., when given at response after a delay). Although priors had a minor influence on encoding, the bulk of the effects were at decision-making. Furthermore, these effects appeared to be distinct. The effect on decision-making was Bayesian-like, with priors inducing bias while improving precision. In contrast, the same priors at encoding improved precision without causing changes in bias. Priors do not just affect encoding or decision-making, but appear to affect both, via distinct mechanisms. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Not all probabilities are equivalent: Evidence from orientation versus spatial probability learning.

Frequently targets are detected faster, probable locations searched earlier, and likely orientations estimated more precisely. Are these all consequences of a single, domain-general "attentional" effect? To examine this issue, participants were shown brief instances of spatial gratings, and were tasked to draw their location and orientation. Unknown to participants, either the location or orientation probability of these gratings were manipulated. While orientation probability affected the precision of orientation reports, spatial probability did not. Further, utilising lowered stimulus contrast (via a staircase procedure) and a combination of behavioral precision and confidence self-report, we clustered trials with perceived stimuli from trials where the target was not detected: Spatial probability only modulated the likelihood of stimulus detection, but not did not modulate perceptual precision. Even when no physical attentional cues are present, acquired probabilistic information on space versus orientation leads to separable 'attention-like' effects on behaviour. We discuss how this could be linked to distinct underlying neural mechanisms. (PsycINFO Database Record

Orientation Probability and Spatial Exogenous Cuing Improve Perceptual Precision and Response Speed by Different Mechanisms.

We are faster and more accurate at detecting frequently occurring objects than infrequent ones, just as we are faster and more accurate at detecting objects that have been spatially cued. Does this behavioral similarity reflect similar processes? To evaluate this question we manipulated orientation probability and exogenous spatial cuing within a single perceptual estimation task. Both increased target probability and spatial cuing led to shorter response initiation times and more precise perceptual reports, but these effects were additive. Further, target probability changed the shape of the distribution of errors while spatial cuing did not. Different routes and independent mechanisms could lead to changes in behavioral measures that look similar to each other and to 'attentional' effects.

Knowing where is different from knowing what: Distinct response time profiles and accuracy effects for target location, orientation, and color probability.

When a location is cued, targets appearing at that location are detected more quickly. When a target feature is cued, targets bearing that feature are detected more quickly. These attentional cueing effects are only superficially similar. More detailed analyses find distinct temporal and accuracy profiles for the two different types of cues. This pattern parallels work with probability manipulations, where both feature and spatial probability are known to affect detection accuracy and reaction times. However, little has been done by way of comparing these effects. Are probability manipulations on space and features distinct? In a series of five experiments, we systematically varied spatial probability and feature probability along two dimensions (orientation or color). In addition, we decomposed response times into initiation and movement components. Targets appearing at the probable location were reported more quickly and more accurately regardless of whether the report was based on orientation or color. On the other hand, when either color probability or orientation probability was manipulated, response time and accuracy improvements were specific for that probable feature dimension. Decomposition of the response time benefits demonstrated that spatial probability only affected initiation times, whereas manipulations of feature probability affected both initiation and movement times. As detection was made more difficult, the two effects further diverged, with spatial probability disproportionally affecting initiation times and feature probability disproportionately affecting accuracy. In conclusion, all manipulations of probability, whether spatial or featural, affect detection. However, only feature probability affects perceptual precision, and precision effects are specific to the probable attribute.

Tuned by experience: How orientation probability modulates early perceptual processing.

Probable stimuli are more often and more quickly detected. While stimulus probability is known to affect decision-making, it can also be explained as a perceptual phenomenon. Using spatial gratings, we have previously shown that probable orientations are also more precisely estimated, even while participants remained naive to the manipulation. We conducted an electrophysiological study to investigate the effect that probability has on perception and visual-evoked potentials. In line with previous studies on oddballs and stimulus prevalence, low-probability orientations were associated with a greater late positive 'P300' component which might be related to either surprise or decision-making. However, the early 'C1' component, thought to reflect V1 processing, was dampened for high-probability orientations while later P1 and N1 components were unaffected. Exploratory analyses revealed a participant-level correlation between C1 and P300 amplitudes, suggesting a link between perceptual processing and decision-making. We discuss how these probability effects could be indicative of sharpening of neurons preferring the probable orientations, due either to perceptual learning, or to feature-based attention.

Probability shapes perceptual precision: A study in orientation estimation.

Probability is known to affect perceptual estimations, but an understanding of mechanisms is lacking. Moving beyond binary classification tasks, we had naive participants report the orientation of briefly viewed gratings where we systematically manipulated contingent probability. Participants rapidly developed faster and more precise estimations for high-probability tilts. The shapes of their error distributions, as indexed by a kurtosis measure, also showed a distortion from Gaussian. This kurtosis metric was robust, capturing probability effects that were graded, contextual, and varying as a function of stimulus orientation. Our data can be understood as a probability-induced reduction in the variability or "shape" of estimation errors, as would be expected if probability affects the perceptual representations. As probability manipulations are an implicit component of many endogenous cuing paradigms, changes at the perceptual level could account for changes in performance that might have traditionally been ascribed to "attention."

The trajectory of the target probability effect.

The effect of target probability on detection times is well-established: Even when detection accuracy is high, lower probability targets are detected more slowly than higher probability ones. Although this target probability effect on detection times has been well-studied, one aspect of it has remained largely unexamined: How the effect develops over the span of an experiment. Here, we investigated this issue with two detection experiments that assessed different target probability ratios. Conventional block segment analysis and linear mixed-effects modeling converged on two key findings. First, we found that the magnitude of the target probability effect increases as one progresses through a block of trials. Second, we found, by examining the trajectories of the low- and high-probability targets, that this increase in effect magnitude was driven by the low-probability targets. Specifically, we found that low-probability targets were detected more slowly as a block of trials progressed. Performance to high-probability targets, on the other hand, was largely invariant across the block. The latter finding is of particular interest because it cannot be reconciled with accounts that propose that the target probability effect is driven by the high-probability targets.