BCI Toolbox: An open-source python package for the Bayesian causal inference model.

Psychological and neuroscientific research over the past two decades has shown that the Bayesian causal inference (BCI) is a potential unifying theory that can account for a wide range of perceptual and sensorimotor processes in humans. Therefore, we introduce the BCI Toolbox, a statistical and analytical tool in Python, enabling researchers to conveniently perform quantitative modeling and analysis of behavioral data. Additionally, we describe the algorithm of the BCI model and test its stability and reliability via parameter recovery. The present BCI toolbox offers a robust platform for BCI model implementation as well as a hands-on tool for learning and understanding the model, facilitating its widespread use and enabling researchers to delve into the data to uncover underlying cognitive mechanisms.

The box-circle illusion.

A novel geometrical optical illusion is reported in this article: the horizontal distances of the contextual structures distort the perceived vertical positions of observed objects. Specifically, the illusion manifests in the form of connected boxes of varying widths but equal heights, each containing a circle at the center. Despite identical vertical positioning of the circles, they appear misaligned. The illusion diminishes when the boxes are removed. Potential underlying mechanisms are discussed.

Perceived Electrical Injury: Misleading Symptomology Due to Multisensory Stimuli.

BACKGROUND: An electrical accident victim's recollection is often distorted by Bayesian inference in multisensory integration. For example, hearing the sound and seeing the bright flash of an electrical arc can create the false impression that someone had experienced an electrical shock. These subjects will often present to an emergency department seeking either treatment or reassurance. CASE REPORTS: We present seven cases in which the subjects were startled by an electrical shock (real or perceived) and injury was reported. Calculations of the current and path were used to allocate causality between the shock and a history of chronic disease or previous trauma. In all seven cases, our analysis suggests that no current was passed through the body. WHY SHOULD AN EMERGENCY PHYSICIAN BE AWARE OF THIS?: Symptomology seen as corroborating may actually be confounding. Witness and survivor descriptions of electrical shocks are fraught with subjectivity and misunderstanding. Available current is usually irrelevant and overemphasized, such as stress on a 100-ampere welding source, which is orders of magnitude beyond lethal limits. History can also be biased for a number of reasons. Bayesian inference in multisensory perception can lead to a subject sincerely believing they had experienced an electrical shock. Determination of the current pathway and calculations of the amplitude and duration of the shock can be critical for understanding the limits and potential causation of electrical injury.

Multisensory causal inference in the brain.

At any given moment, our brain processes multiple inputs from its different sensory modalities (vision, hearing, touch, etc.). In deciphering this array of sensory information, the brain has to solve two problems: (1) which of the inputs originate from the same object and should be integrated and (2) for the sensations originating from the same object, how best to integrate them. Recent behavioural studies suggest that the human brain solves these problems using optimal probabilistic inference, known as Bayesian causal inference. However, how and where the underlying computations are carried out in the brain have remained unknown. By combining neuroimaging-based decoding techniques and computational modelling of behavioural data, a new study now sheds light on how multisensory causal inference maps onto specific brain areas. The results suggest that the complexity of neural computations increases along the visual hierarchy and link specific components of the causal inference process with specific visual and parietal regions.

A biologically inspired neurocomputational model for audiovisual integration and causal inference.

Recently, experimental and theoretical research has focused on the brain's abilities to extract information from a noisy sensory environment and how cross-modal inputs are processed to solve the causal inference problem to provide the best estimate of external events. Despite the empirical evidence suggesting that the nervous system uses a statistically optimal and probabilistic approach in addressing these problems, little is known about the brain's architecture needed to implement these computations. The aim of this work was to realize a mathematical model, based on physiologically plausible hypotheses, to analyze the neural mechanisms underlying multisensory perception and causal inference. The model consists of three layers topologically organized: two encode auditory and visual stimuli, separately, and are reciprocally connected via excitatory synapses and send excitatory connections to the third downstream layer. This synaptic organization realizes two mechanisms of cross-modal interactions: the first is responsible for the sensory representation of the external stimuli, while the second solves the causal inference problem. We tested the network by comparing its results to behavioral data reported in the literature. Among others, the network can account for the ventriloquism illusion, the pattern of sensory bias and the percept of unity as a function of the spatial auditory-visual distance, and the dependence of the auditory error on the causal inference. Finally, simulations results are consistent with probability matching as the perceptual strategy used in auditory-visual spatial localization tasks, agreeing with the behavioral data. The model makes untested predictions that can be investigated in future behavioral experiments.

Effects of tempo, swing density, and listener's drumming experience, on swing detection thresholds for drum rhythms.

Swing, a popular technique in music performance, has been said to enhance the "groove" of the rhythm. Swing works by delaying the onsets of even-numbered subdivisions of each beat (e.g., 16th-note swing delays the onsets of the second and fourth 16th-note subdivisions of each quarter-note beat). The "swing magnitude" (loosely speaking, the amount of delay) is often quite small. And there has been little investigation, using musical stimuli, into what swing magnitudes listeners can detect. To that end, this study presented continually-looped electronic drum rhythms, with 16th-note swing in the hi-hat on every other bar, to drummers and non-drummers. Swing magnitude was adjusted using a staircase procedure, to determine the magnitude where the difference between swinging and not-swinging bars was just-noticeable. Different tempi (60 to 140 quarter-notes per minute) and swing densities (how often notes occurred at even-numbered subdivisions) were used. Results showed that all subjects could detect smaller swing magnitudes when swing density was higher, thus confirming a previous speculation that the perceptual salience of swing increases with swing density. The just-noticeable magnitudes of swing for drummers differed from those of non-drummers, in terms of both overall magnitude and sensitivity to tempo, thus prompting questions for further exploration.

The Brain's Tendency to Bind Audiovisual Signals Is Stable but Not General.

Previous studies have shown a surprising amount of between-subjects variability in the strength of interactions between sensory modalities. For the same set of stimuli, some subjects exhibit strong interactions, whereas others exhibit weak interactions. To date, little is known about what underlies this variability. Sensory integration in the brain could be governed by a global mechanism or by task-specific mechanisms that could be either stable or variable across time. We used a rigorous quantitative tool (Bayesian causal inference) to investigate whether integration (i.e., binding) tendencies generalize across tasks and are stable across time. We report for the first time that individuals' binding tendencies are stable across time but are task-specific. These results provide evidence against the hypothesis that sensory integration is governed by a single, global parameter in the brain.

Biases in Visual, Auditory, and Audiovisual Perception of Space.

Localization of objects and events in the environment is critical for survival, as many perceptual and motor tasks rely on estimation of spatial location. Therefore, it seems reasonable to assume that spatial localizations should generally be accurate. Curiously, some previous studies have reported biases in visual and auditory localizations, but these studies have used small sample sizes and the results have been mixed. Therefore, it is not clear (1) if the reported biases in localization responses are real (or due to outliers, sampling bias, or other factors), and (2) whether these putative biases reflect a bias in sensory representations of space or a priori expectations (which may be due to the experimental setup, instructions, or distribution of stimuli). Here, to address these questions, a dataset of unprecedented size (obtained from 384 observers) was analyzed to examine presence, direction, and magnitude of sensory biases, and quantitative computational modeling was used to probe the underlying mechanism(s) driving these effects. Data revealed that, on average, observers were biased towards the center when localizing visual stimuli, and biased towards the periphery when localizing auditory stimuli. Moreover, quantitative analysis using a Bayesian Causal Inference framework suggests that while pre-existing spatial biases for central locations exert some influence, biases in the sensory representations of both visual and auditory space are necessary to fully explain the behavioral data. How are these opposing visual and auditory biases reconciled in conditions in which both auditory and visual stimuli are produced by a single event? Potentially, the bias in one modality could dominate, or the biases could interact/cancel out. The data revealed that when integration occurred in these conditions, the visual bias dominated, but the magnitude of this bias was reduced compared to unisensory conditions. Therefore, multisensory integration not only improves the precision of perceptual estimates, but also the accuracy.

Clarifying signal detection theoretic interpretations of the Müller-Lyer and sound-induced flash illusions.

In a recent article, Witt, Taylor, Sugovic, and Wixted (2015) made several claims about the way the Müller-Lyer and sound-induced flash illusions should influence the signal detection theory criterion and sensitivity measures, c and d'. Here, we address some crucial conceptual inconsistencies in their simulation of the Müller-Lyer illusion and clarify a previous analysis of the sound-induced flash illusion from the literature that is misinterpreted in their discussion. Alternative signal detection theoretic interpretations of both illusions are offered.

Perception, as you make it.

The main question that Firestone & Scholl (F&S) pose is whether "what and how we see is functionally independent from what and how we think, know, desire, act, and so forth" (sect. 2, para. 1). We synthesize a collection of concerns from an interdisciplinary set of coauthors regarding F&S's assumptions and appeals to intuition, resulting in their treatment of visual perception as context-free.

The effects of color and saturation on the enjoyment of real-life images.

This study investigated the effects of color presence and saturation on the affective judgment of real-life images, as functions of the image's affective valence. In the first two experiments, participants observed and rated original color photos and their grayscale versions, presented in an interleaved order across two separate experimental sessions. Color photos were rated as more pleasant than grayscale photos when image valence was positive, and more unpleasant when image valence was negative. The third experiment consisted of the same original images and their versions with saturation reduced by 50%. Original photos were rated as more pleasant than saturation-reduced photos when image valence was positive, yet less unpleasant when image valence was negative, implying potentially separable mechanisms for processing color presence and saturation. Significant interactions were found between color or saturation mode and valence on affective judgment in all three experiments. The effects persisted after controlling for colorfulness and luminance between the color and grayscale (or de-saturated) conditions.

The overlooked role of unisensory precision in multisensory research.

Zhu et al. present an alternative explanation for the weaker multisensory illusions in football goalkeepers compared with outfielders and non-athletes, showing that better unisensory precision in goalkeepers can also account for this effect.

Crossmodal interactions in human learning and memory.

Most studies of memory and perceptual learning in humans have employed unisensory settings to simplify the study paradigm. However, in daily life we are often surrounded by complex and cluttered scenes made up of many objects and sources of sensory stimulation. Our experiences are, therefore, highly multisensory both when passively observing the world and when acting and navigating. We argue that human learning and memory systems are evolved to operate under these multisensory and dynamic conditions. The nervous system exploits the rich array of sensory inputs in this process, is sensitive to the relationship between the sensory inputs, and continuously updates sensory representations, and encodes memory traces based on the relationship between the senses. We review some recent findings that demonstrate a range of human learning and memory phenomena in which the interactions between visual and auditory modalities play an important role, and suggest possible neural mechanisms that can underlie some surprising recent findings. We outline open questions as well as directions of future research to unravel human perceptual learning and memory.

Big number, big body: Jersey numbers alter body size perception.

Vision has been shown to be an active process that can be shaped by top-down influences. Here, we add to this area of research by showing a surprising example of how visual perception can be affected by cognition (i.e., cognitive penetration). Observers were presented, on each trial, with a picture of a computer-generated football player and asked to rate the slenderness of the player on an analog scale. The results of two experiments showed that observers perceived athletes wearing small jersey numbers as more slender than those with high numbers. This finding suggests that the cognition of numbers quantitatively alters body size perception. We conjecture that this effect is the result of previously learned associations (i.e., prior expectations) affecting perceptual inference. Such associations are likely the result of implicit learning of the statistical regularities of number and size attributes co-occurrences by the nervous system. We discuss how these results are consistent with previous research on statistical learning and how they fit into the Bayesian framework of perception. The current finding supports the notion of top-down influences of cognition on perception.

Improving memory for unusual events with wakeful reactivation.

Memory consists of multiple processes, from encoding information, consolidating it into short- and long- term memory, and later retrieving relevant information. Targeted memory reactivation is an experimental method during which sensory components of a multisensory representation (such as sounds or odors) are 'reactivated', facilitating the later retrieval of unisensory attributes. We examined whether novel and unpredicted events benefit from reactivation to a greater degree than normal stimuli. We presented participants with everyday objects, and 'tagged' these objects with sounds (e.g., animals and their matching sounds) at different screen locations. 'Oddballs' were created by presenting unusual objects and sounds (e.g., a unicorn with a heartbeat sound). During a short reactivation phase, participants listened to a replay of normal and oddball sounds. Participants were then tested on their memory for visual and spatial information in the absence of sounds. Participants were better at remembering the oddball objects compared to normal ones. Importantly, participants were also better at recalling the locations of oddball objects whose sounds were reactivated, compared to objects whose sounds that were not presented again. These results suggest that episodic memory benefits from associating objects with unusual cues, and that reactivating those cues strengthen the entire multisensory representation, resulting in enhanced memory for unisensory attributes.

Recalibration of auditory space following milliseconds of cross-modal discrepancy.

Basic features of objects and events in the environment such as timing and spatial location are encoded by multiple sensory modalities. This redundancy in sensory coding allows recalibration of one sense by other senses if there is a conflict between the sensory maps (Radeau and Bertelson, 1974; Zwiers et al., 2003; Navarra et al., 2009). In contrast to motor or sensorimotor adaptation, which can be relatively rapid, cross-sensory recalibration (the change in an isolated sensory representation after exposure to conflicting cross-modal information) has been reported only as a result of an extensive amount of exposure to sensory discrepancy (e.g., hundreds or thousands of trials, or prolonged durations). Therefore, sensory recalibration has traditionally been associated with compensation for permanent changes that would occur during development or after traumatic injuries or stroke. Nonetheless, the dynamics of sensory recalibration is unknown, and it is unclear whether prolonged inconsistency is required to trigger recalibration or whether such mechanisms are continuously engaged in self-maintenance. We show that in humans recalibration of perceived auditory space by vision can occur after a single exposure to discrepant auditory-visual stimuli lasting only a few milliseconds. These findings suggest an impressive degree of plasticity in a basic perceptual map induced by a cross-modal error signal. Therefore, it appears that modification of sensory maps does not necessarily require accumulation of a substantial amount of evidence of error to be triggered, and is continuously operational. This scheme of sensory recalibration has many advantages. It only requires a small working memory capacity, and allows rapid adaptation to transient changes in the environment as well as the body.

0 + 1 > 1: How adding noninformative sound improves performance on a visual task.

It is well known that the nervous system combines information from different cues within and across sensory modalities to improve performance on perceptual tasks. In this article, we present results showing that in a visual motion-detection task, concurrent auditory motion stimuli improve accuracy even when they do not provide any useful information for the task. When participants judged which of two stimulus intervals contained visual coherent motion, the addition of identical moving sounds to both intervals improved accuracy. However, this enhancement occurred only with sounds that moved in the same direction as the visual motion. Therefore, it appears that the observed benefit of auditory stimulation is due to auditory-visual interactions at a sensory level. Thus, auditory and visual motion-processing pathways interact at a sensory-representation level in addition to the level at which perceptual estimates are combined.

Bayesian causal inference: A unifying neuroscience theory.

Understanding of the brain and the principles governing neural processing requires theories that are parsimonious, can account for a diverse set of phenomena, and can make testable predictions. Here, we review the theory of Bayesian causal inference, which has been tested, refined, and extended in a variety of tasks in humans and other primates by several research groups. Bayesian causal inference is normative and has explained human behavior in a vast number of tasks including unisensory and multisensory perceptual tasks, sensorimotor, and motor tasks, and has accounted for counter-intuitive findings. The theory has made novel predictions that have been tested and confirmed empirically, and recent studies have started to map its algorithms and neural implementation in the human brain. The parsimony, the diversity of the phenomena that the theory has explained, and its illuminating brain function at all three of Marr's levels of analysis make Bayesian causal inference a strong neuroscience theory. This also highlights the importance of collaborative and multi-disciplinary research for the development of new theories in neuroscience.

Subliminal audio-visual temporal congruency in music videos enhances perceptual pleasure.

Human perception is inherently multisensory, with cross-modal integration playing a critical role in generating a coherent perceptual experience. To understand the causes of pleasurable experiences, we must understand whether and how the relationship between separate sensory modalities influences our experience of pleasure. We investigated the effect of congruency between vision and audition in the form of temporal alignment between the cuts in a video and the beats in an accompanying soundtrack. Despite the subliminal nature of this manipulation, a higher perceptual pleasure was found for temporal congruency compared with incongruency. These results suggest that the temporal aspect of the interaction between the visual and auditory modalities plays a critical role in shaping our perceptual pleasure, even when such interaction is not accessible to conscious awareness.