Reduced perceptual narrowing in synesthesia.

Synesthesia is a neurologic trait in which specific inducers, such as sounds, automatically elicit additional idiosyncratic percepts, such as color (thus "colored hearing"). One explanation for this trait-and the one tested here-is that synesthesia results from unusually weak pruning of cortical synaptic hyperconnectivity during early perceptual development. We tested the prediction from this hypothesis that synesthetes would be superior at making discriminations from nonnative categories that are normally weakened by experience-dependent pruning during a critical period early in development-namely, discrimination among nonnative phonemes (Hindi retroflex /d̪a/ and dental /ɖa/), among chimpanzee faces, and among inverted human faces. Like the superiority of 6-mo-old infants over older infants, the synesthetic groups were significantly better than control groups at making all the nonnative discriminations across five samples and three testing sites. The consistent superiority of the synesthetic groups in making discriminations that are normally eliminated during infancy suggests that residual cortical connectivity in synesthesia supports changes in perception that extend beyond the specific synesthetic percepts, consistent with the incomplete pruning hypothesis.

Learning at Variable Attentional Load Requires Cooperation of Working Memory, Meta-learning, and Attention-augmented Reinforcement Learning.

Flexible learning of changing reward contingencies can be realized with different strategies. A fast learning strategy involves using working memory of recently rewarded objects to guide choices. A slower learning strategy uses prediction errors to gradually update value expectations to improve choices. How the fast and slow strategies work together in scenarios with real-world stimulus complexity is not well known. Here, we aim to disentangle their relative contributions in rhesus monkeys while they learned the relevance of object features at variable attentional load. We found that learning behavior across six monkeys is consistently best predicted with a model combining (i) fast working memory and (ii) slower reinforcement learning from differently weighted positive and negative prediction errors as well as (iii) selective suppression of nonchosen feature values and (iv) a meta-learning mechanism that enhances exploration rates based on a memory trace of recent errors. The optimal model parameter settings suggest that these mechanisms cooperate differently at low and high attentional loads. Whereas working memory was essential for efficient learning at lower attentional loads, enhanced weighting of negative prediction errors and meta-learning were essential for efficient learning at higher attentional loads. Together, these findings pinpoint a canonical set of learning mechanisms and suggest how they may cooperate when subjects flexibly adjust to environments with variable real-world attentional demands.

Quaddles: A multidimensional 3-D object set with parametrically controlled and customizable features.

Many studies of vision and cognition require novel three-dimensional object sets defined by a parametric feature space. Creating such sets and verifying that they are suitable for a given task, however, can be time-consuming and effortful. Here we present a new set of multidimensional objects, Quaddles, designed for studies of feature-based learning and attention, but adaptable for many research purposes. Quaddles have features that are all equally visible from any angle around the vertical axis and can be designed to be equally discriminable along feature dimensions; these objects do not show strong or consistent response biases, with a small number of quantified exceptions. They are available as two-dimensional images, rotating videos, and FBX object files suitable for use with any modern video game engine. We also provide scripts that can be used to generate hundreds of thousands of further Quaddles, as well as examples and tutorials for modifying Quaddles or creating completely new object sets from scratch, with the aim to speed up the development time of future novel-object studies.

The prevalence of synaesthesia depends on early language learning.

According to one theory, synaesthesia develops, or is preserved, because it helps children learn. If so, it should be more common among adults who faced greater childhood learning challenges. In the largest survey of synaesthesia to date, the incidence of synaesthesia was compared among native speakers of languages with transparent (easier) and opaque (more difficult) orthographies. Contrary to our prediction, native speakers of Czech (transparent) were more likely to be synaesthetes than native speakers of English (opaque). However, exploratory analyses suggested that this was because more Czechs learned non-native second languages, which was strongly associated with synaesthesia, consistent with the learning hypothesis. Furthermore, the incidence of synaesthesia among speakers of opaque languages was double that among speakers of transparent languages other than Czech, also consistent with the learning hypothesis. These findings contribute to an emerging understanding of synaesthetic development as a complex and lengthy process with multiple causal influences.

Fixations are not all created equal: An objection to mindless visual search.

This call to revolution in theories of visual search does not go far enough. Treating fixations as uniform is an oversimplification that obscures the critical role of the mind. We remind readers that what happens during a fixation depends on mindset, as shown in studies of search strategy and of humans' ability to rapidly resume search following an interruption.

A Multi-task Platform for Profiling Cognitive and Motivational Constructs in Humans and Nonhuman Primates.

Background: Understanding the neurobiological substrates of psychiatric disorders requires comprehensive evaluations of cognitive and motivational functions in preclinical research settings. The translational validity of such evaluations will be supported by (1) tasks with high construct validity that are engaging and easy to teach to human and nonhuman participants, (2) software that enables efficient switching between multiple tasks in single sessions, (3) software that supports tasks across a broad range of physical experimental setups, and (4) by platform architectures that are easily extendable and customizable to encourage future optimization and development. New Method: We describe the Multi-task Universal Suite for Experiments (M-USE), a software platform designed to meet these requirements. It leverages the Unity video game engine and C# programming language to (1) support immersive and engaging tasks for humans and nonhuman primates, (2) allow experimenters or participants to switch between multiple tasks within-session, (3) generate builds that function across computers, tablets, and websites, and (4) is freely available online with documentation and tutorials for users and developers. M-USE includes a task library with seven pre-existing tasks assessing cognitive and motivational constructs of perception, attention, working memory, cognitive flexibility, motivational and affective self-control, relational long-term memory, and visuo-spatial problem solving. Results: M-USE was used to test NHPs on up to six tasks per session, all available as part of the Task Library, and to extract performance metrics for all major cognitive and motivational constructs spanning the Research Domain Criteria (RDoC) of the National Institutes of Mental Health. Comparison with Existing Methods: Other experiment design and control systems exist, but do not provide the full range of features available in M-USE, including a pre-existing task library for cross-species assessments; the ability to switch seamlessly between tasks in individual sessions; cross-platform build capabilities; license-free availability; and its leveraging of video-engine capabilities used to gamify tasks. Conclusions: The new multi-task platform facilitates cross-species translational research for understanding the neurobiological substrates of higher cognitive and motivational functions.

Grapheme-color synaesthesia benefits rule-based Category learning.

Researchers have long suspected that grapheme-color synaesthesia is useful, but research on its utility has so far focused primarily on episodic memory and perceptual discrimination. Here we ask whether it can be harnessed during rule-based Category learning. Participants learned through trial and error to classify grapheme pairs that were organized into categories on the basis of their associated synaesthetic colors. The performance of synaesthetes was similar to non-synaesthetes viewing graphemes that were physically colored in the same way. Specifically, synaesthetes learned to categorize stimuli effectively, they were able to transfer this learning to novel stimuli, and they falsely recognized grapheme-pair foils, all like non-synaesthetes viewing colored graphemes. These findings demonstrate that synaesthesia can be exploited when learning the kind of material taught in many classroom settings.

The Curve Visible on the Campbell-Robson Chart Is Not the Contrast Sensitivity Function.

The Campbell-Robson chart is a highly popular figure used in psychophysics and visual perception textbooks to illustrate the Contrast Sensitivity Function (CSF). The chart depicts a grating which varies logarithmically in spatial frequency (SF) from left to right and in contrast from bottom to top. Campbell and Robson's (1964) intuition was that the boundary between the grating and the homogeneous gray area (below threshold) would trace the shape of the observer's own CSF. In this paper, we tested this intuition. A total of 170 participants (96 adults and 74 children) adjusted the four parameters of a truncated log-parabola directly onto a Campbell-Robson chart rendition and completed a gold-standard CSF evaluation. We hoped that this procedure which requires a mere three clicks on the computer mouse, would speed up the measurement of the CSF to under a minute. Unfortunately, the only parameter of the truncated log-parabola fitted to the gold-standard CSF data that could be predicted from the Campbell-Robson chart data was the peak sensitivity for the adult participants. We conclude that the curve visible on the Campbell-Robson chart cannot be used practically to measure the CSF.

A Kiosk Station for the Assessment of Multiple Cognitive Domains and Cognitive Enrichment of Monkeys.

Nonhuman primates (NHP's) are self-motivated to perform cognitive tasks on touchscreens in their animal housing setting. To leverage this ability, fully integrated hardware and software solutions are needed that work within housing and husbandry routines while also spanning cognitive task constructs of the Research Domain Criteria (RDoC). Here, we detail such an integrated robust hardware and software solution for running cognitive tasks in cage-housed NHP's with a cage-mounted Kiosk Station (KS-1). KS-1 consists of a frame for mounting flexibly on housing cages, a touchscreen animal interface with mounts for receptables, reward pumps, and cameras, and a compact computer cabinet with an interface for controlling behavior. Behavioral control is achieved with a Unity3D program that is virtual-reality capable, allowing semi-naturalistic visual tasks to assess multiple cognitive domains.KS-1 is fully integrated into the regular housing routines of monkeys. A single person can operate multiple KS-1's. Monkeys engage with KS-1 at high motivation and cognitive performance levels at high intra-individual consistency. KS-1 is optimized for flexible mounting onto standard apartment cage systems and provides a new design variation complementing existing cage-mounted touchscreen systems. KS-1 has a robust animal interface with options for gaze/reach monitoring. It has an integrated user interface for controlling multiple cognitive tasks using a common naturalistic object space designed to enhance task engagement. All custom KS-1 components are open-sourced.In summary, KS-1 is a versatile new tool for cognitive profiling and cognitive enrichment of cage-housed monkeys. It reliably measures multiple cognitive domains which promises to advance our understanding of animal cognition, inter-individual differences, and underlying neurobiology in refined, ethologically meaningful behavioral foraging contexts.

MAD saccade: statistically robust saccade threshold estimation via the median absolute deviation.

Saccade detection is a critical step in the analysis of gaze data. A common method for saccade detection is to use a simple threshold for velocity or acceleration values, which can be estimated from the data using the mean and standard deviation. However, this method has the downside of being influenced by the very signal it is trying to detect, the outlying velocities or accelerations that occur during saccades. We propose instead to use the median absolute deviation (MAD), a robust estimator of dispersion that is not influenced by outliers. We modify an algorithm proposed by Nyström and colleagues, and quantify saccade detection performance in both simulated and human data. Our modified algorithm shows a significant and marked improvement in saccade detection - showing both more true positives and less false negatives - especially under higher noise levels. We conclude that robust estimators can be widely adopted in other common, automatic gaze classification algorithms due to their ease of implementation.

Extremely selective attention: eye-tracking studies of the dynamic allocation of attention to stimulus features in categorization.

Humans have an extremely flexible ability to categorize regularities in their environment, in part because of attentional systems that allow them to focus on important perceptual information. In formal theories of categorization, attention is typically modeled with weights that selectively bias the processing of stimulus features. These theories make differing predictions about the degree of flexibility with which attention can be deployed in response to stimulus properties. Results from 2 eye-tracking studies show that humans can rapidly learn to differently allocate attention to members of different categories. These results provide the first unequivocal demonstration of stimulus-responsive attention in a categorization task. Furthermore, the authors found clear temporal patterns in the shifting of attention within trials that follow from the informativeness of particular stimulus features. These data provide new insights into the attention processes involved in categorization.

USE: An integrative suite for temporally-precise psychophysical experiments in virtual environments for human, nonhuman, and artificially intelligent agents.

BACKGROUND: There is a growing interest in complex, active, and immersive behavioral neuroscience tasks. However, the development and control of such tasks present unique challenges. NEW METHOD: The Unified Suite for Experiments (USE) is an integrated set of hardware and software tools for the design and control of behavioral neuroscience experiments. The software, developed using the Unity video game engine, supports both active tasks in immersive 3D environments and static 2D tasks used in more traditional visual experiments. The custom USE SyncBox hardware, based around an Arduino Mega2560 board, integrates and synchronizes multiple data streams from different pieces of experimental hardware. The suite addresses three key issues with developing cognitive neuroscience experiments in Unity: tight experimental control, accurate sub-ms timing, and accurate gaze target identification. RESULTS: USE is a flexible framework to realize experiments, enabling (i) nested control over complex tasks, (ii) flexible use of 3D or 2D scenes and objects, (iii) touchscreen-, button-, joystick- and gaze-based interaction, and (v) complete offline reconstruction of experiments for post-processing and temporal alignment of data streams. COMPARISON WITH EXISTING METHODS: Most existing experiment-creation tools are not designed to support the development of video-game-like tasks. Those that do use older or less popular video game engines as their base, and are not as feature-rich or enable as precise control over timing as USE. CONCLUSIONS: USE provides an integrated, open source framework for a wide variety of active behavioral neuroscience experiments using human and nonhuman participants, and artificially-intelligent agents.

Changes of Attention during Value-Based Reversal Learning Are Tracked by N2pc and Feedback-Related Negativity.

Previously learned reward values can have a pronounced impact, behaviorally and neurophysiologically, on the allocation of selective attention. All else constant, stimuli previously associated with a high value gain stronger attentional prioritization than stimuli previously associated with a low value. The N2pc, an ERP component indicative of attentional target selection, has been shown to reflect aspects of this prioritization, by changes of mean amplitudes closely corresponding to selective enhancement of high value target processing and suppression of high value distractor processing. What has remained unclear so far is whether the N2pc also reflects the flexible and repeated behavioral adjustments needed in a volatile task environment, in which the values of stimuli are reversed often and unannounced. Using a value-based reversal learning task, we found evidence that the N2pc amplitude flexibly and reversibly tracks value-based choices during the learning of reward associated stimulus colors. Specifically, successful learning of current value-contingencies was associated with reduced N2pc amplitudes, and this effect was more apparent for distractor processing, compared with target processing. In addition, following a value reversal the feedback related negativity(FRN), an ERP component that reflects feedback processing, was amplified and co-occurred with increased N2pc amplitudes in trials following low-value feedback. Importantly, participants that showed the greatest adjustment in N2pc amplitudes based on feedback were also the most efficient learners. These results allow further insight into how changes in attentional prioritization in an uncertain and volatile environment support flexible adjustments of behavior.

Synesthesia and learning: a critical review and novel theory.

Learning and synesthesia are profoundly interconnected. On the one hand, the development of synesthesia is clearly influenced by learning. Synesthetic inducers - the stimuli that evoke these unusual experiences - often involve the perception of complex properties learned in early childhood, e.g., letters, musical notes, numbers, months of the year, and even swimming strokes. Further, recent research has shown that the associations individual synesthetes make with these learned inducers are not arbitrary, but are strongly influenced by the structure of the learned domain. For instance, the synesthetic colors of letters are partially determined by letter frequency and the relative positions of letters in the alphabet. On the other hand, there is also a small, but growing, body of literature which shows that synesthesia can influence or be helpful in learning. For instance, synesthetes appear to be able to use their unusual experiences as mnemonic devices and can even exploit them while learning novel abstract categories. Here we review these two directions of influence and argue that they are interconnected. We propose that synesthesia arises, at least in part, because of the cognitive demands of learning in childhood, and that it is used to aid perception and understanding of a variety of learned categories. Our thesis is that the structural similarities between synesthetic triggering stimuli and synesthetic experiences are the remnants, the fossilized traces, of past learning challenges for which synsethesia was helpful.

Temporal characteristics of overt attentional behavior during category learning.

Many theories of category learning incorporate mechanisms for selective attention, typically implemented as attention weights that change on a trial-by-trial basis. This is because there is relatively little data on within-trial changes in attention. We used eye tracking and mouse tracking as fine-grained measures of attention in three complex visual categorization tasks to investigate temporal patterns in overt attentional behavior within individual categorization decisions. In Experiments 1 and 2, we recorded participants' eye movements while they performed three different categorization tasks. We extended previous research by demonstrating that not only are participants less likely to fixate irrelevant features, but also, when they do, these fixations are shorter than fixations to relevant features. We also found that participants' fixation patterns show increasingly consistent temporal patterns. Participants were faster, although no more accurate, when their fixation sequences followed a consistent temporal structure. In Experiment 3, we replicated these findings in a task where participants used mouse movements to uncover features. Overall, we showed that there are important temporal regularities in information sampling during category learning that cannot be accounted for by existing models. These can be used to supplement extant models for richer predictions of how information is attended to during the buildup to a categorization decision.

Second-order mappings in grapheme-color synesthesia.

Typically, the search for order in grapheme-color synesthesia has been conducted by looking at the frequency of certain letter-color associations. Here, we report stronger associations when second-order similarity mappings are examined--specifically, mappings between the synesthetic colors of letters and letter shape, frequency, and position in the alphabet. The analyses demonstrate that these relations are independent of one other. More strikingly, our analyses show that each of the letter-color mappings is restricted to one dimension of color, with letter shape and ordinality linked to hue, and letter frequency linked to luminance. These results imply that synesthetic associations are acquired as the alphabet is learned, with associations involving letter shape, ordinality, and frequency being made independently and idiosyncratically. Because these mappings of similarity structure between domains (letters and colors) are similar to those found in numerous other cognitive and perceptual domains, they imply that synesthetic associations operate on principles common to many aspects of human cognition.

Person perception informs understanding of cognition during visual search.

Does person perception--the impressions we form from watching others--hold clues to the mental states of people engaged in cognitive tasks? We investigated this with a two-phase method: In Phase 1, participants searched on a computer screen (Experiment 1) or in an office (Experiment 2); in Phase 2, other participants rated the searchers' video-recorded behavior. The results showed that blind raters are sensitive to individual differences in search proficiency and search strategy, as well as to environmental factors affecting search difficulty. Also, different behaviors were linked to search success in each setting: Eye movement frequency predicted successful search on a computer screen; head movement frequency predicted search success in an office. In both settings, an active search strategy and positive emotional expressions were linked to search success. These data indicate that person perception informs cognition beyond the scope of performance measures, offering the potential for new measurements of cognition that are both rich and unobtrusive.

Looking versus seeing: Strategies alter eye movements during visual search.

Visual search can be made more efficient by adopting a passive cognitive strategy (i.e., letting the target "pop" into mind) rather than by trying to actively guide attention. In the present study, we examined how this strategic benefit is linked to eye movements. Results show that participants using a passive strategy wait longer before beginning to move their eyes and make fewer saccades than do active participants. Moreover, the passive advantage stems from more efficient use of the information in a fixation, rather than from a wider attentional window. Individual difference analyses indicate that strategies also change the way eye movements are related to search success, with a rapid saccade rate predicting success among active participants, and fewer and larger amplitude saccades predicting success among passive participants. A change in mindset, therefore, alters how oculomotor behaviors are harnessed in the service of visual search.

Errors, efficiency, and the interplay between attention and category learning.

Learning to identify objects as members of categories is an essential cognitive skill and learning to deploy attention effectively is a core component of that process. The present study investigated an assumption imbedded in formal models of categorization: error is necessary for attentional learning. Eye-trackers were used to record participants' allocation of attention to task relevant and irrelevant features while learning a complex categorization task. It was found that participants optimized their fixation patterns in the absence of both performance errors and corrective external feedback. Optimization began immediately after each category was mastered and continued for many trials. These results demonstrate that error is neither necessary nor sufficient for all forms of attentional learning.