A novel model of divergent predictive perception.

Predictive processing theories state that our subjective experience of reality is shaped by a balance of expectations based on previous knowledge about the world (i.e. priors) and confidence in sensory input from the environment. Divergent experiences (e.g. hallucinations and synaesthesia) are likely to occur when there is an imbalance between one's reliance on priors and sensory input. In a novel theoretical model, inspired by both predictive processing and psychological principles, we propose that predictable divergent experiences are associated with natural or environmentally induced prior/sensory imbalances: inappropriately strong or inflexible (i.e. maladaptive) high-level priors (beliefs) combined with low sensory confidence can result in reality discrimination issues, a characteristic of psychosis; maladaptive low-level priors (sensory expectations) combined with high sensory confidence can result in atypical sensory sensitivities and persistent divergent percepts, a characteristic of synaesthesia. Crucially, we propose that whether different divergent experiences manifest with dominantly sensory (e.g. hallucinations) or nonsensory characteristics (e.g. delusions) depends on mental imagery ability, which is a spectrum from aphantasia (absent or weak imagery) to hyperphantasia (extremely vivid imagery). We theorize that imagery is critically involved in shaping the sensory richness of divergent perceptual experience. In sum, to predict a range of divergent perceptual experiences in both clinical and general populations, three factors must be accounted for: a maladaptive use of priors, individual level of confidence in sensory input, and mental imagery ability. These ideas can be expressed formally using nonparametric regression modeling. We provide evidence for our theory from previous work and deliver predictions for future research.

Genetic and environmental architecture of synaesthesia and its association with the autism spectrum-a twin study.

Synaesthesia is a sensory phenomenon where external stimuli, such as sounds or letters, trigger additional sensations (e.g. colours). Synaesthesia aggregates in families but its heritability is unknown. The phenomenon is more common in people on the autism spectrum compared with the general population and associated with higher autistic traits. Using classical twin design, we assessed the heritability of individual differences in self-reported synaesthesia and the genetic and environmental contributions to their association with autistic traits within a population twin cohort (n = 4262, age = 18 years). We estimated individual differences in synaesthesia to be heritable and influenced by environmental factors not shared between twins. The association between individual differences in synaesthesia and autistic traits was estimated to be predominantly under genetic influence and seemed to be mainly driven by non-social autistic traits (repetitive behaviours, restricted interests and attention to detail). Our study suggests that the link between synaesthesia and autism might reside in shared genetic causes, related to non-social autistic traits such as alterations in perception. Future studies building on these findings may attempt to identify specific groups of genes that influence both autism, synaesthesia and perception.

synr: An R package for handling synesthesia consistency test data.

Synesthesia is a phenomenon where sensory stimuli or cognitive concepts elicit additional perceptual experiences. For instance, in a commonly studied type of synesthesia, stimuli such as words written in black font elicit experiences of other colors, e.g., red. In order to objectively verify synesthesia, participants are asked to choose colors for repeatedly presented stimuli and the consistency of their choices is evaluated (consistency test). Previously, there has been no publicly available and easy-to-use tool for analyzing consistency test results. Here, the R package synr is introduced, which provides an efficient interface for exploring consistency test data and applying common procedures for analyzing them. Importantly, synr also implements a novel method enabling identification of participants whose scores cannot be interpreted, e.g., who only give black or red color responses. To this end, density-based spatial clustering of applications with noise (DBSCAN) is applied in conjunction with a measure of spread in 3D space. An application of synr with pre-existing openly accessible data illustrating how synr is used in practice is presented. Also included is a comparison of synr's data validation procedure and human ratings, which found that synr had high correspondence with human ratings and outperformed human raters in situations where human raters were easily mislead. Challenges for widespread adoption of synr as well as suggestions for using synr within the field of synesthesia and other areas of psychological research are discussed.

Perceptual processing links autism and synesthesia: A co-twin control study.

Synesthesia occurs more commonly in individuals fulfilling criteria for an autism spectrum diagnosis than in the general population. It is associated with autistic traits and autism-related perceptual processing characteristics, including a more detail-focused attentional style and altered sensory sensitivity. In addition, these characteristics correlate with the degree of grapheme-color synesthesia (consistency of grapheme-color associations) in non-synesthetes. We investigated a predominantly non-synesthetic twin sample, including individuals fulfilling criteria for an autism spectrum diagnosis or other neurodevelopmental disorders (n = 65, 14-34 years, 60% female). We modelled linear relationships between the degree of grapheme-color synesthesia and autistic traits, sensory sensitivity, and visual perception, both within-twin pairs (22 pairs) where all factors shared by twins are implicitly controlled (including 50-100% genetics), and across the entire cohort. We found that the degree of grapheme-color synesthesia was associated with autistic traits within the domain of Attention to Details and with sensory hyper-, but not hypo-sensitivity. These associations were stronger within-twin pairs than across the sample. Further, twins with a higher degree of grapheme-color synesthesia were better than their co-twins at identifying fragmented images (Fragmented Pictures Test). This is the first twin study on the association between synesthesia and autism-related perceptual features and traits. The results suggest that investigating these associations within-twin pairs, implicitly adjusting for potential confounding factors shared by twins, is more sensitive than doing so in non-related individuals. Consistent with previous findings, the results suggest an association between the degree of grapheme-color synesthesia and autism-related perceptual features, while utilizing a different measure for sensory sensitivity. The novel finding of enhanced fragmented picture integration in twins with a higher degree of grapheme-color synesthesia challenges the view of a generally more detail-focused attentional style in synesthesia and might be related to enhanced memory or mental imagery in more synesthetic individuals.

Perceptual Gains and Losses in Synesthesia and Schizophrenia.

Individual differences in perception are widespread. Considering inter-individual variability, synesthetes experience stable additional sensations; schizophrenia patients suffer perceptual deficits in, eg, perceptual organization (alongside hallucinations and delusions). Is there a unifying principle explaining inter-individual variability in perception? There is good reason to believe perceptual experience results from inferential processes whereby sensory evidence is weighted by prior knowledge about the world. Perceptual variability may result from different precision weighting of sensory evidence and prior knowledge. We tested this hypothesis by comparing visibility thresholds in a perceptual hysteresis task across medicated schizophrenia patients (N = 20), synesthetes (N = 20), and controls (N = 26). Participants rated the subjective visibility of stimuli embedded in noise while we parametrically manipulated the availability of sensory evidence. Additionally, precise long-term priors in synesthetes were leveraged by presenting either synesthesia-inducing or neutral stimuli. Schizophrenia patients showed increased visibility thresholds, consistent with overreliance on sensory evidence. In contrast, synesthetes exhibited lowered thresholds exclusively for synesthesia-inducing stimuli suggesting high-precision long-term priors. Additionally, in both synesthetes and schizophrenia patients explicit, short-term priors-introduced during the hysteresis experiment-lowered thresholds but did not normalize perception. Our results imply that perceptual variability might result from differences in the precision afforded to prior beliefs and sensory evidence, respectively.

Synaesthesia and autism: Different developmental outcomes from overlapping mechanisms?

Synaesthesia, a mixing of the senses, is more common in individuals with autism. Here, we review the evidence for the association between synaesthesia and autism with regard to their genetic background, brain connectivity, perception, cognitive mechanisms and their contribution to exceptional talents. Currently, the overlap between synaesthesia and autism is established most convincingly at the level of alterations in sensory sensitivity and perception, with synaesthetes showing autism-like profiles of sensory sensitivity and a bias towards details in perception. Shared features may include a predominance of local over global connectivity in the brain. When autism and synaesthesia co-occur in the same individual, the chance of developing heightened cognitive and memory abilities is increased. We discuss how the same theoretical models could potentially explain both conditions. Given the evidence, we believe the phenotypical overlap between autism and synaesthesia has been established clearly enough to invite future research to confirm overlapping mechanisms.

Enhanced spatial navigation skills in sequence-space synesthetes.

Individuals with sequence-space synesthesia (SSS) perceive sequences like months, days and numbers in certain spatial arrangements. Several cognitive benefits have been associated with SSS, such as enhanced mental rotation, more vivid visual imagery and an advantage in spatial processing. The current study aimed to further investigate these cognitive benefits, focusing on spatial navigation skills, to explore if their enhanced sensitivity to spatial relations is reflected in enhanced navigational performance. Synesthetes were distinguished from controls by means of a questionnaire, a consistency test and drawings. A virtual Morris Water Maze (MWM) task with two allocentric and two egocentric navigation conditions was used to assess spatial navigation abilities. For the allocentric tasks, participants had to use object cues to find a hidden platform and for the egocentric tasks, they had to use their own position as a reference. Results showed that synesthetes performed significantly better compared to controls on the allocentric and egocentric tasks that reflected real life situations more accurately. However, this significant result was only found for the time taken to find the platform and not for the length of the path that was taken. In exploratory analyses, no significant relations were found between task performance and the specific features of the manifestation of each individual's synesthesia. Our hypothesis that synesthetes with the ability to mentally rotate their spatial arrangements would perform better on the allocentric task was not confirmed. Results add to the growing body of literature concerning the cognitive benefits of SSS and are consistent with the possibility that enhanced spatial navigation skills emerge from generally enhanced visuospatial abilities in SSS.

The Relation Between Autistic Traits, the Degree of Synaesthesia, and Local/Global Visual Perception.

Synaesthesia is highly prevalent in autism spectrum disorder. We assessed the relation between the degree of autistic traits (Autism Spectrum Quotient, AQ) and the degree of synaesthesia in a neurotypical population, and hypothesized both are related to a local bias in visual perception. A positive correlation between total AQ scores and the degree of synaesthesia was found, extending previous studies in clinical populations. Consistent with our hypothesis, AQ-attention to detail scores were related to increased performance on an Embedded Figures Task and reduced susceptibility to visual illusions. We found no relation between autistic traits and performance on a motion coherence task, and no relation between synaesthesia and local visual perception. Possibly, this relation is reserved for supra-threshold synaesthetes.

Evidence for confounding eye movements under attempted fixation and active viewing in cognitive neuroscience.

Eye movements can have serious confounding effects in cognitive neuroscience experiments. Therefore, participants are commonly asked to fixate. Regardless, participants will make so-called fixational eye movements under attempted fixation, which are thought to be necessary to prevent perceptual fading. Neural changes related to these eye movements could potentially explain previously reported neural decoding and neuroimaging results under attempted fixation. In previous work, under attempted fixation and passive viewing, we found no evidence for systematic eye movements. Here, however, we show that participants' eye movements are systematic under attempted fixation when active viewing is demanded by the task. Since eye movements directly affect early visual cortex activity, commonly used for neural decoding, our findings imply alternative explanations for previously reported results in neural decoding.

Autistic traits in synaesthesia: atypical sensory sensitivity and enhanced perception of details.

In synaesthetes, specific sensory stimuli (e.g. black letters) elicit additional experiences (e.g. colour). Synaesthesia is highly prevalent among individuals with autism spectrum disorder (ASD), but the mechanisms of this co-occurrence are not clear. We hypothesized autism and synaesthesia share atypical sensory sensitivity and perception. We assessed autistic traits, sensory sensitivity and visual perception in two synaesthete populations. In Study 1, synaesthetes (N = 79, of different types) scored higher than non-synaesthetes (N = 76) on the Attention-to-detail and Social skills subscales of the autism spectrum quotient indexing autistic traits, and on the Glasgow Sensory Questionnaire indexing sensory hypersensitivity and hyposensitivity which frequently occur in autism. Synaesthetes performed two local/global visual tasks because individuals with autism typically show a bias towards detail processing. In synaesthetes, elevated motion coherence thresholds (MCTs) suggested reduced global motion perception, and higher accuracy on an embedded figures task suggested enhanced local perception. In Study 2, sequence-space synaesthetes (N = 18) completed the same tasks. Questionnaire and embedded figures results qualitatively resembled Study 1 results, but no significant group differences with non-synaesthetes (N = 20) were obtained. Unexpectedly, sequence-space synaesthetes had reduced MCTs. Altogether, our studies suggest atypical sensory sensitivity and a bias towards detail processing are shared features of synaesthesia and ASD. This article is part of the discussion meeting issue 'Bridging senses: novel insights from synaesthesia'.

Investigating genetic links between grapheme-colour synaesthesia and neuropsychiatric traits.

Synaesthesia is a neurological phenomenon affecting perception, where triggering stimuli (e.g. letters and numbers) elicit unusual secondary sensory experiences (e.g. colours). Family-based studies point to a role for genetic factors in the development of this trait. However, the contributions of common genomic variation to synaesthesia have not yet been investigated. Here, we present the SynGenes cohort, the largest genotyped collection of unrelated people with grapheme-colour synaesthesia (n = 723). Synaesthesia has been associated with a range of other neuropsychological traits, including enhanced memory and mental imagery, as well as greater sensory sensitivity. Motivated by the prior literature on putative trait overlaps, we investigated polygenic scores derived from published genome-wide scans of schizophrenia and autism spectrum disorder (ASD), comparing our SynGenes cohort to 2181 non-synaesthetic controls. We found a very slight association between schizophrenia polygenic scores and synaesthesia (Nagelkerke's R2 = 0.0047, empirical p = 0.0027) and no significant association for scores related to ASD (Nagelkerke's R2 = 0.00092, empirical p = 0.54) or body mass index (R2 = 0.00058, empirical p = 0.60), included as a negative control. As sample sizes for studying common genomic variation continue to increase, genetic investigations of the kind reported here may yield novel insights into the shared biology between synaesthesia and other traits, to complement findings from neuropsychology and brain imaging. This article is part of a discussion meeting issue 'Bridging senses: novel insights from synaesthesia'.

Enhanced self-reported affect and prosocial behaviour without differential physiological responses in mirror-sensory synaesthesia.

Mirror-sensory synaesthetes mirror the pain or touch that they observe in other people on their own bodies. This type of synaesthesia has been associated with enhanced empathy. We investigated whether the enhanced empathy of people with mirror-sensory synesthesia influences the experience of situations involving touch or pain and whether it affects their prosocial decision making. Mirror-sensory synaesthetes (N = 18, all female), verified with a touch-interference paradigm, were compared with a similar number of age-matched control individuals (all female). Participants viewed arousing images depicting pain or touch; we recorded subjective valence and arousal ratings, and physiological responses, hypothesizing more extreme reactions in synaesthetes. The subjective impact of positive and negative images was stronger in synaesthetes than in control participants; the stronger the reported synaesthesia, the more extreme the picture ratings. However, there was no evidence for differential physiological or hormonal responses to arousing pictures. Prosocial decision making was assessed with an economic game assessing altruism, in which participants had to divide money between themselves and a second player. Mirror-sensory synaesthetes donated more money than non-synaesthetes, showing enhanced prosocial behaviour, and also scored higher on the Interpersonal Reactivity Index as a measure of empathy. Our study demonstrates the subjective impact of mirror-sensory synaesthesia and its stimulating influence on prosocial behaviour. This article is part of the discussion meeting issue 'Bridging senses: novel insights from synaesthesia'.

Cross-modal associations and synesthesia: Categorical perception and structure in vowel-color mappings in a large online sample.

We report associations between vowel sounds, graphemes, and colors collected online from over 1,000 Dutch speakers. We also provide open materials, including a Python implementation of the structure measure and code for a single-page web application to run simple cross-modal tasks. We also provide a full dataset of color-vowel associations from 1,164 participants, including over 200 synesthetes identified using consistency measures. Our analysis reveals salient patterns in the cross-modal associations and introduces a novel measure of isomorphism in cross-modal mappings. We found that, while the acoustic features of vowels significantly predict certain mappings (replicating prior work), both vowel phoneme category and grapheme category are even better predictors of color choice. Phoneme category is the best predictor of color choice overall, pointing to the importance of phonological representations in addition to acoustic cues. Generally, high/front vowels are lighter, more green, and more yellow than low/back vowels. Synesthetes respond more strongly on some dimensions, choosing lighter and more yellow colors for high and mid front vowels than do nonsynesthetes. We also present a novel measure of cross-modal mappings adapted from ecology, which uses a simulated distribution of mappings to measure the extent to which participants' actual mappings are structured isomorphically across modalities. Synesthetes have mappings that tend to be more structured than nonsynesthetes', and more consistent color choices across trials correlate with higher structure scores. Nevertheless, the large majority (~ 70%) of participants produce structured mappings, indicating that the capacity to make isomorphically structured mappings across distinct modalities is shared to a large extent, even if the exact nature of the mappings varies across individuals. Overall, this novel structure measure suggests a distribution of structured cross-modal association in the population, with synesthetes at one extreme and participants with unstructured associations at the other.

Neuroimaging Findings on Amodal Completion: A Review.

Amodal completion is the phenomenon of perceiving completed objects even though physically they are partially occluded. In this review, we provide an extensive overview of the results obtained from a variety of neuroimaging studies on the neural correlates of amodal completion. We discuss whether low-level and high-level cortical areas are implicated in amodal completion; provide an overview of how amodal completion unfolds over time while dissociating feedforward, recurrent, and feedback processes; and discuss how amodal completion is represented at the neuronal level. The involvement of low-level visual areas such as V1 and V2 is not yet clear, while several high-level structures such as the lateral occipital complex and fusiform face area seem invariant to occlusion of objects and faces, respectively, and several motor areas seem to code for object permanence. The variety of results on the timing of amodal completion hints to a mixture of feedforward, recurrent, and feedback processes. We discuss whether the invisible parts of the occluded object are represented as if they were visible, contrary to a high-level representation. While plenty of questions on amodal completion remain, this review presents an overview of the neuroimaging findings reported to date, summarizes several insights from computational models, and connects research of other perceptual completion processes such as modal completion. In all, it is suggested that amodal completion is the solution to deal with various types of incomplete retinal information, and highly depends on stimulus complexity and saliency, and therefore also give rise to a variety of observed neural patterns.

Nonrandom Associations of Graphemes with Colors in Arabic.

Numerous studies demonstrate people associate colors with letters and numbers in systematic ways. But most of these studies rely on speakers of English, or closely related languages. This makes it difficult to know how generalizable these findings are, or what factors might underlie these associations. We investigated letter-color and number-color associations in Arabic speakers, who have a different writing system and unusual word structure compared to Standard Average European languages. We also aimed to identify grapheme-color synaesthetes (people who have conscious color experiences with letters and numbers). Participants associated colors with 28 basic Arabic letters and ten digits by typing color names that best fit each grapheme. We found language-specific principles determining grapheme-color associations. For example, the word formation process in Arabic was relevant for color associations. In addition, psycholinguistic variables, such as letter frequency and the intrinsic order of graphemes influenced associations. Contrary to previous studies we found no evidence for sounds playing a role in letter-color associations for Arabic, and only a very limited role for shape influencing color associations. These findings highlight the importance of linguistic and psycholinguistic features in cross-modal correspondences, and illustrate why it is important to play close attention to each language on its own terms in order to disentangle language-specific from universal effects.

The Merit of Synesthesia for Consciousness Research.

Synesthesia is a phenomenon in which additional perceptual experiences are elicited by sensory stimuli or cognitive concepts. Synesthetes possess a unique type of phenomenal experiences not directly triggered by sensory stimulation. Therefore, for better understanding of consciousness it is relevant to identify the mental and physiological processes that subserve synesthetic experience. In the present work we suggest several reasons why synesthesia has merit for research on consciousness. We first review the research on the dynamic and rapidly growing field of the studies of synesthesia. We particularly draw attention to the role of semantics in synesthesia, which is important for establishing synesthetic associations in the brain. We then propose that the interplay between semantics and sensory input in synesthesia can be helpful for the study of the neural correlates of consciousness, especially when making use of ambiguous stimuli for inducing synesthesia. Finally, synesthesia-related alterations of brain networks and functional connectivity can be of merit for the study of consciousness.

Constructing priors in synesthesia.

A new theoretical framework (PPSMC) applicable to synesthesia has been proposed, in which the discrepancy between the perceptual reality of (some) synesthetic concurrents and their subjective non-veridicality is being explained. The PPSMC framework stresses the relevance of the phenomenology of synesthesia for synesthesia research-and beyond. When describing the emergence and persistence of synesthetic concurrents under PPSMC, it is proposed that precise, high-confidence priors are crucial in synesthesia. I discuss the construction of priors in synesthesia.

Phonological markers of information structure: an fMRI study.

In this fMRI study we investigate the neural correlates of information structure integration during sentence comprehension in Dutch. We looked into how prosodic cues (pitch accents) that signal the information status of constituents to the listener (new information) are combined with other types of information during the unification process. The difficulty of unifying the prosodic cues into overall sentence meaning was manipulated by constructing sentences in which the pitch accent did (focus-accent agreement), and sentences in which the pitch accent did not (focus-accent disagreement) match the expectations for focus constituents of the sentence. In case of a mismatch, the load on unification processes increases. Our results show two anatomically distinct effects of focus-accent disagreement, one located in the posterior left inferior frontal gyrus (LIFG, BA6/44), and one in the more anterior-ventral LIFG (BA 47/45). Our results confirm that information structure is taken into account during unification, and imply an important role for the LIFG in unification processes, in line with previous fMRI studies.