Reward impacts visual statistical learning.

Humans automatically detect and remember regularities in the visual environment-a type of learning termed visual statistical learning (VSL). Many aspects of learning from reward resemble VSL in certain respects, yet whether and how reward learning impacts VSL is largely unexamined. In two studies, we found that reward contingencies affect VSL, with high-value associated with stronger behavioral and neural signatures of such learning than low-value images. In Experiment 1, participants learned values (high or low) of images through a trial-and-error risky choice task. Unbeknownst to them, images were paired as four types-High-High, High-Low, Low-High, and Low-Low. In subsequent recognition and reward memory tests, participants chose the more familiar of two pairs (a target and a foil) and recalled the value of images. We found better recognition when the first images of pairs have high-values, with High-High pairs showing the highest recognition rate. In Experiment 2, we provided evidence that both value and statistical contingencies affected brain responses. When we compared responses between the high-value first image and the low-value first image, greater activation in regions that included inferior frontal gyrus, anterior cingulate gyrus, hippocampus, among other regions, were found. These findings were driven by the interaction between statistically structured information and reward-the same value contrast yielded no regions for second-image contrasts and for singletons. Our results suggest that when reward information is embedded in stimulus-stimulus associations, it may alter the learning process; specifically, the higher-value first image potentially enables better memory for statistically learned pairs and reward information.

Object-based warping in three-dimensional environments.

Object-based warping is a powerful visual illusion wherein space between features within figural regions is regularly overestimated compared with those within ground regions. Originally, the effect was only examined in displays of two-dimensional (2D) stimuli. The present study sought to examine whether object-based warping persists in more naturalistic viewing conditions, where additional contextual cues are present. Stimuli were presented with either three-dimensional (3D) printed objects (Experiment 1) or 3D objects in virtual reality (Experiments 2-4). The testing metric was actual distance of features (dots) compared with estimated distances made by participants. Responses for the 3D printed stimuli were measured with replica dots on a slide ruler device. The virtual reality experiments collected responses either with a computer mouse or motion-tracked controller and included manipulations of object type, spatial separation, viewing distance of stimuli, and head motion. A standard warping effect in 3D was observed in all experiments, although the effect was not present in one condition that elicits warping in 2D (Occluded Rectangle). The final experiment resolves this discrepancy by reducing the multicomponent object (Occluded Rectangle) to a single component figure, while demonstrating the influence of depth cues on the warping effect under occlusion. Collectively, these experiments reveal that object-based warping is a powerful effect, even in naturalistic settings.

Not-so-working Memory: Drift in Functional Magnetic Resonance Imaging Pattern Representations during Maintenance Predicts Errors in a Visual Working Memory Task.

Working memory (WM) is critical to many aspects of cognition, but it frequently fails. Much WM research has focused on capacity limits, but even for single, simple features, the fidelity of individual representations is limited. Why is this? One possibility is that, because of neural noise and interference, neural representations do not remain stable across a WM delay, nor do they simply decay, but instead, they may "drift" over time to a new, less accurate state. We tested this hypothesis in a functional magnetic resonance imaging study of a match/nonmatch WM recognition task for a single item with a single critical feature: orientation. We developed a novel pattern-based index of "representational drift" to characterize ongoing changes in brain activity patterns throughout the WM maintenance period, and we were successfully able to predict performance on the match/nonmatch recognition task using this representational drift index. Specifically, in trials where the target and probe stimuli matched, participants incorrectly reported more nonmatches when their activity patterns drifted away from the target. In trials where the target and probe did not match, participants incorrectly reported more matches when their activity patterns drifted toward the probe. On the basis of these results, we contend that neural noise does not cause WM errors merely by degrading representations and increasing random guessing; instead, one means by which noise introduces errors is by pushing WM representations away from the target and toward other meaningful (yet incorrect) configurations. Thus, we demonstrate that behaviorally meaningful drift within representation space can be indexed by neuroimaging.

Visual statistical learning deficits in memory-impaired individuals.

Visual statistical learning (VSL) refers to the learning of environmental regularities. Classically considered an implicit process, one patient with isolated hippocampal damage is severely impaired at VSL tasks, suggesting involvement of explicit memory. Here, we asked whether memory impairment (MI) alone, absent of clear hippocampal pathology, predicted deficits across different VSL tasks. A classic VSL task revealed no learning in MI participants (Exp. 1), while imposing attentional demands (Exp. 2: flicker detection, Exp. 3: gender/location categorization) during familiarization revealed modest residual VSL. MI with nonspecific neural correlates predicted impaired VSL overall, but attentional processes may be harnessed for rehabilitation.

Opponent Identity Influences Value Learning in Simple Games.

Context plays a pivotal role in many decision-making scenarios, including social interactions wherein the identities and strategies of other decision makers often shape our behaviors. However, the neural mechanisms for tracking such contextual information are poorly understood. Here, we investigated how opponent identity affects human reinforcement learning during a simulated competitive game against two independent computerized opponents. We found that strategies of participants were affected preferentially by the outcomes of the previous interactions with the same opponent. In addition, reinforcement signals from the previous trial were less discriminable throughout the brain after the opponent changed, compared with when the same opponent was repeated. These opponent-selective reinforcement signals were particularly robust in right rostral anterior cingulate and right lingual regions, where opponent-selective reinforcement signals correlated with a behavioral measure of opponent-selective reinforcement learning. Therefore, when choices involve multiple contextual frames, such as different opponents in a game, decision making and its neural correlates are influenced by multithreaded histories of reinforcement. Overall, our findings are consistent with the availability of temporally overlapping, context-specific reinforcement signals. SIGNIFICANCE STATEMENT: In real-world decision making, context plays a strong role in determining the value of an action. Similar choices take on different values depending on setting. We examined the contextual dependence of reward-based learning and reinforcement signals using a simple two-choice matching-pennies game played by humans against two independent computer opponents that were randomly interleaved. We found that human subjects' strategies were highly dependent on opponent context in this game, a fact that was reflected in select brain regions' activity (rostral anterior cingulate and lingual cortex). These results indicate that human reinforcement histories are highly dependent on contextual factors, a fact that is reflected in neural correlates of reinforcement signals.

Expansion and compression of space within and beyond the boundaries of an object.

Objects illusorily distort our perception of space, as indexed by perceived distances between two reference points placed within an object compared with the perceived distances between two dots placed in a ground region. This paper examines several novel aspects of such distortions across three experiments that employed a staircase procedure to determine the point of subjective equivalence between dot distances for one pair of dots within or near an object, compared to dots that were placed on a ground region. We replicate and expand upon prior findings that showed that dots within an object's boundaries are perceived as further apart than they are - an expansion effect. We also verify and quantify a subjective experience that was previously unreported - dots positioned within the object but near or on the boundaries, as well as dots positioned beyond the extent of an object's boundaries, are perceived as closer together (compressed) versus ground-region dots. We additionally demonstrate that expansion and compression extend into space laterally adjacent to an object. These findings demonstrate novel properties of the impact of objects on the perception of spatial relationships, and place important constraints on potential mechanisms that could explain object-based warping.

Visual statistical learning is modulated by arbitrary and natural categories.

Visual statistical learning (VSL) describes the unintentional extraction of statistical regularities from visual environments across time or space, and is typically studied using novel stimuli (e.g., symbols unfamiliar to participants) and using familiarization procedures that are passive or require only basic vigilance. The natural visual world, however, is rich with a variety of complex visual stimuli, and we experience that world in the presence of goal-driven behavior including overt learning of other kinds. To examine how VSL responds to such contexts, we exposed subjects to statistical contingencies as they learned arbitrary categorical mappings of unfamiliar stimuli (fractals, Experiment 1) or familiar stimuli with preexisting categorical boundaries (faces and scenes, Experiment 2). In a familiarization stage, subjects learned by trial and error the arbitrary mappings between stimuli and one of two responses. Unbeknownst to participants, items were paired such that they always appeared together in the stream. Pairs were equally likely to be of the same or different category. In a pair recognition stage to assess VSL, subjects chose between a target pair and a foil pair. In both experiments, subjects' VSL was shaped by arbitrary categories: same-category pairs were learned better than different-category pairs. Natural categories (Experiment 2) also played a role, with subjects learning same-natural-category pairs at higher rates than different-category pairs, an effect that did not interact with arbitrary mappings. We conclude that learning goals of the observer and preexisting knowledge about the structure of the world play powerful roles in the incidental learning of novel statistical information.

Object-based warping: an illusory distortion of space within objects.

Visual objects are high-level primitives that are fundamental to numerous perceptual functions, such as guidance of attention. We report that objects warp visual perception of space in such a way that spatial distances within objects appear to be larger than spatial distances in ground regions. When two dots were placed inside a rectangular object, they appeared farther apart from one another than two dots with identical spacing outside of the object. To investigate whether this effect was object based, we measured the distortion while manipulating the structure surrounding the dots. Object displays were constructed with a single object, multiple objects, a partially occluded object, and an illusory object. Nonobject displays were constructed to be comparable to object displays in low-level visual attributes. In all cases, the object displays resulted in a more powerful distortion of spatial perception than comparable non-object-based displays. These results suggest that perception of space within objects is warped.

Inferior parietal lobule supports decision making under uncertainty in humans.

The optimal responses for many decisions faced by humans are ill defined, yet we are able to choose well by associating choices with outcomes, and employing this information in decision making. Previous studies suggest that the parietal cortex is involved in "uncertain" decision making, yet uncertainty is confounded with increased difficulty and attention. Here we aim to dissociate the role of parietal cortex in decision making and attention. Using functional magnetic resonance imaging we measured brain activity while participants played a "matching-pennies" game. We found that the inferior parietal lobule is involved in decision making under uncertainty, showing higher activity when the decision was uncertain rather than certain and when humans were given trial-by-trial feedback on choice outcomes than when they were not. Crucially, increasing attentional load with secondary tasks reduced inferior parietal activity when decisions were made under uncertainty, suggesting that general attention does not drive its activation. This pattern was consistent for visual or auditory feedback, and for direct (symbols representing wins and losses) or indirect (only the opponent's choices were shown) feedback. It contrasted with results from medial superior frontal gyrus, which was driven primarily by increased attentional load. We suggest that decision making under uncertainty is dissociable from general attention in the brain.

Oculomotor capture reveals trial-by-trial neural correlates of attentional guidance by contents of visual working memory.

Evidence from attentional and oculomotor capture, contingent capture, and other paradigms suggests that mechanisms supporting human visual working memory (VWM) and visual attention are intertwined. Features held in VWM bias guidance toward matching items even when those features are task irrelevant. However, the neural basis of this interaction is underspecified. Prior examinations using fMRI have primarily relied on coarse comparisons across experimental conditions that produce varying amounts of capture. To examine the neural dynamics of attentional capture on a trial-by-trial basis, we applied an oculomotor paradigm that produced discrete measures of capture. On each trial, subjects were shown a memory item, followed by a blank retention interval, then a saccade target that appeared to the left or right. On some trials, an irrelevant distractor appeared above or below fixation. Once the saccade target was fixated, subjects completed a forced-choice memory test. Critically, either the target or distractor could match the feature held in VWM. Although task irrelevant, this manipulation produced differences in behavior: participants were more likely to saccade first to an irrelevant VWM-matching distractor compared with a non-matching distractor - providing a discrete measure of capture. We replicated this finding while recording eye movements and scanning participants' brains using fMRI. To examine the neural basis of oculomotor capture, we separately modeled the retention interval for capture and non-capture trials within the distractor-match condition. We found that frontal activity, including anterior cingulate cortex and superior frontal gyrus regions, differentially predicted subsequent oculomotor capture by a memory-matching distractor. Other regions previously implicated as involved in attentional capture by VWM-matching items showed no differential activity across capture and non-capture trials, even at a liberal threshold. Our findings demonstrate the power of trial-by-trial analyses of oculomotor capture as a means to examine the underlying relationship between VWM and attentional guidance systems.

Supercrowding: weakly masking a target expands the range of crowding.

Crowding is impairment of peripheral object identification by nearby objects. Critical spacing (the minimum target-flanker distance that does not produce crowding) scales with target eccentricity and is consistently reported as roughly equal to or less than 50% of target eccentricity (0.5e). This study demonstrates that crowding occurs far beyond the typical critical spacing when the target is weakly masked by a surrounding contour or backwards pattern mask. A target was presented at a peripheral location on every trial and participants reported its orientation. Flankers appeared at target-flanker distances of 0.3-0.7e, or were absent. The target was presented with or without a mask. When flankers were absent, the masks only mildly impaired performance. When flankers were present but the mask was absent, target identification was nearly perfect at wide target-flanker distances (0.5e-0.7e). However, when flankers were present and the target was masked, performance dropped significantly, even when target-flanker distances far exceeded the typical crowding range. This phenomenon ("supercrowding") shares critical features with standard crowding: flankers similar to the target impair performance more than dissimilar flankers, and the characteristic anisotropic profile of crowding is preserved. Supercrowding may reflect a general interaction between crowding and other forms of masking.

Multiple states in visual working memory: Evidence from oculomotor capture by memory-matching distractors.

Visual working memory (VWM) representations interact with attentional guidance, but there is controversy over whether multiple VWM items simultaneously influence attentional guidance. Extant studies relied on continuous variables like response times, which can obscure capture - especially if VWM representations cycle through interactive and non-interactive states. Previous conflicting findings regarding guidance when under high working memory (WM) load may be due to the use of noisier response time measures that mix capture and non-capture trials. Thus, we employed an oculomotor paradigm to characterize discrete attentional capture events under both high and low VWM load. Participants held one or two colors in memory, then executed a saccade to a target disk. On some trials, a distractor (sometimes VWM-matching) appeared simultaneously with the target. Eye movements were more frequently directed to a VWM-matching than a non-matching distractor for both load conditions. However, oculomotor capture by a VWM-matching distractor occurred less frequently under high compared with low load. These results suggest that attention is automatically guided toward items matching only one of two colors held in memory at a time, suggesting that items in VWM may cycle through attention-guiding and not-guiding states when more than one item is held in VWM and the task does not require that multiple items be maintained in an active, attention-guiding state.

Individual differences in object recognition.

There is substantial evidence for individual differences in personality and cognitive abilities, but we lack clear intuitions about individual differences in visual abilities. Previous work on this topic has typically compared performance with only 2 categories, each measured with only 1 task. This approach is insufficient for demonstration of domain-general effects. Most previous work has used familiar object categories, for which experience may vary between participants and categories, thereby reducing correlations that would stem from a common factor. In Study 1, we adopted a latent variable approach to test for the first time whether there is a domain-general object recognition ability, o. We assessed whether shared variance between latent factors representing performance for each of 5 novel object categories could be accounted for by a single higher-order factor. On average, 89% of the variance of lower-order factors denoting performance on novel object categories could be accounted for by a higher-order factor, providing strong evidence for o. Moreover, o also accounted for a moderate proportion of variance in tests of familiar object recognition. In Study 2, we assessed whether the strong association across categories in object recognition is due to third-variable influences. We find that o has weak to moderate associations with a host of cognitive, perceptual, and personality constructs and that a clear majority of the variance in and covariance between performance on different categories is independent of fluid intelligence. This work provides the first demonstration of a reliable, specific, and domain-general object recognition ability, and suggest a rich framework for future work in this area. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Induced perceptual grouping.

The term perceptual grouping is associated with classical principles such as similarity and proximity. This article reports induced perceptual grouping, a phenomenon that occurs when placement of a uniform set of items near a structured set induces grouping within the otherwise uniform set. For example, when items grouped pair-wise by similarity are placed near another set of unstructured items, an analogous pair-wise grouping links elements of the second set. Induced grouping affected reaction times in two different visual search tasks, with reaction times depending on whether the target properties were contained within a group or crossed group boundaries as defined by induced grouping due to similarity, proximity, or common fate. Induced grouping was reduced when grouping between the structured and unstructured sets was weakened by means of a common-region cue or decreased similarity. Induced grouping appears to reflect the computation of hierarchical structure in visual images.

The roles of order, distance, and interstitial items in temporal visual statistical learning.

Humans are adept at learning regularities in a visual environment, even without explicit cues to structure and in the absence of instruction-this has been termed "visual statistical learning" (VSL). The nature of the representations resulting from VSL are still poorly understood. In five experiments, we examined the specificity of temporal VSL representations. In Experiments 1A, 1B, and 2, we compared recognition rates of triplets and all embedded pairs to chance. Robust learning of all structures was evident, and even pairs of non-adjacent items in a sequentially presented triplet (AC extracted from a triplet composed of ABC) were recognized at above-chance levels. In Experiment 3, we asked whether people could recognize rearranged pairs to examine the flexibility of learned representations. Recognition of all possible orders of target triplets and pairs was significantly higher than chance, and there were no differences between canonical orderings and their corresponding randomized orderings, suggesting that learners were not dependent upon originally experienced stimulus orderings to recognize co-occurrence. Experiment 4 demonstrates the essential role of an interstitial item in VSL representations. By comparing the learning of quadruplet sets (e.g., ABCD) and triplet sets (e.g., ABC), we found learning of AC and BD in ABCD (quadruplet) sets were better than the learning of AC in ABC (triplet) sets. This pattern of results might result from the critical role of interstitial items in statistical learning. In short, our work supports the idea of generalized representation in VSL and provides evidence about how this representation is structured.

Tasks determine what is learned in visual statistical learning.

Visual statistical learning (VSL), the unsupervised learning of statistical contingencies across time and space, may play a key role in efficient and predictive encoding of the perceptual world. How VSL capabilities vary as a function of ongoing task demands is still poorly understood. VSL is modulated by selective attention and faces interference from some secondary tasks, but there is little evidence that the types of contingencies learned in VSL are sensitive to task demands. We found a powerful effect of task on what is learned in VSL. Participants first completed a visual familiarization task requiring judgments of face gender (female/male) or scene location (interior/exterior). Statistical regularities were embedded between stimulus pairs. During a surprise recognition phase, participants showed less recognition for pairs that had required a change in response key (e.g., female followed by male) or task (e.g., female followed by indoor) during familiarization. When familiarization required detection of "flicker" or "jiggle" events unrelated to image content, there was weaker, but uniform, VSL across pair types. These results suggest that simple task manipulations play a strong role in modulating the distribution of learning over different pair combinations. Such variations may arise from task and response conflict or because the manner in which images are processed is altered.

Self-relevance effects and label choice: Strong variations in label-matching performance due to non-self-relevant factors.

Merely associating one's self with a stimulus may be enough to enhance performance in a label-matching paradigm (Sui, He, & Humphreys, 2012), implying prioritized processing of self-relevant stimuli. For instance, labeling a square as SELF and a circle as OTHER yields speeded performance when verifying square-SELF compared with circle-OTHER label matches. The precise causes of such effects are unclear. We propose that prioritized processing of label-matches can occur for reasons other than self-relevance. Here, we employ the label-matching paradigm to show similar benefits for non-self-relevant labels (SNAKE, FROG, and GREG) over a frequently employed, non-self-relevant control label (OTHER). These benefits suggest the possibility that self-relevance effects in the label-matching paradigm may be confounded with other properties of labels that lead to relative performance benefits, such as concreteness. The size of self-relevance effects may be overestimated in prior work employing the label-matching paradigm, which calls for greater care in the choice of control labels to determine the true magnitude of self-relevance effects. Our results additionally indicate the possibility of a powerful effect of concreteness (and related properties) on associative memory performance.

Associating resting-state connectivity with trait impulsivity.

Psychometric research has identified stable traits that predict inter-individual differences in appetitive motivation and approach behavior. Behavioral Inhibition System/Behavioral Activation System (BIS/BAS) scales have been developed to quantitatively assess these traits. However, neural mechanisms corresponding to the proposed constructs reflected in BIS/BAS are still poorly defined. The ventral striatum (VS) and orbitofrontal cortex (OFC) are implicated in subserving reward-related functions that are also associated with the BAS. In this study, we examined whether functional connectivity between these regions predicts components of these scales. We employed resting-state functional connectivity and BIS/BAS scores assessed by a personality questionnaire. Participants completed a resting state run and the Behavioral Inhibition and Activation Systems (BIS/BAS) Questionnaire. Using resting-state BOLD, we assessed correlations between two basal ganglia ROIs (caudate and putamen) and bilateral OFC ROIs, establishing single subject connectivity summary scores. Summary scores were correlated with components of BIS/BAS scores. Results demonstrate a novel correlation between BAS-fun seeking and resting-state connectivity between middle OFC and putamen, implying that spontaneous synchrony between reward-processing regions may play a role in defining personality characteristics related to impulsivity.

No Apparent Influence of Reward upon Visual Statistical Learning.

Humans are capable of detecting and exploiting a variety of environmental regularities, including stimulus-stimulus contingencies (e.g., visual statistical learning) and stimulus-reward contingencies. However, the relationship between these two types of learning is poorly understood. In two experiments, we sought evidence that the occurrence of rewarding events enhances or impairs visual statistical learning. Across all of our attempts to find such evidence, we employed a training stage during which we grouped shapes into triplets and presented triplets one shape at a time in an undifferentiated stream. Participants subsequently performed a surprise recognition task in which they were tested on their knowledge of the underlying structure of the triplets. Unbeknownst to participants, triplets were also assigned no-, low-, or high-reward status. In Experiments 1A and 1B, participants viewed shape streams while low and high rewards were "randomly" given, presented as low- and high-pitched tones played through headphones. Rewards were always given on the third shape of a triplet (Experiment 1A) or the first shape of a triplet (Experiment 1B), and high- and low-reward sounds were always consistently paired with the same triplets. Experiment 2 was similar to Experiment 1, except that participants were required to learn value associations of a subset of shapes before viewing the shape stream. Across all experiments, we observed significant visual statistical learning effects, but the strength of learning did not differ amongst no-, low-, or high-reward conditions for any of the experiments. Thus, our experiments failed to find any influence of rewards on statistical learning, implying that visual statistical learning may be unaffected by the occurrence of reward. The system that detects basic stimulus-stimulus regularities may operate independently of the system that detects reward contingencies.

Integrating sequential arrays in visual short-term memory.

Are sequential visual arrays represented as separate images or as a combined image in visual short-term memory (VSTM)? Proponents of the integration account suggest that an image of the first array is gradually formed and integrated with an image of the second to produce a combined representation. This view is evidenced by successful performance in an empty-cell detection task. In this task, on a 4 x 4 square matrix, 7 locations are occupied on a first array, followed by a variable interval, and then by 8 other occupied locations on a second array. Subjects' success in identifying the remaining empty cell has been taken as evidence for integration. In this study, we show that success in this task can be better accounted for by a convert-and-compare process than by an integration process. We conclude that VSTM only supports limited integration across sequential arrays.