Multisensory-inspired modeling and neural correlates for two key binocular interactions.

Most binocular vision models assume that the two eyes sum incompletely. However, some facilitatory cortical neurons fire for only one eye, but amplify their firing rates if both eyes are stimulated. These 'binocular gate' neurons closely resemble subthreshold multisensory neurons. Binocular amplification for binocular gate neurons follows a power law, with a compressive exponent. Unexpectedly, this rule also applies to facilitatory true binocular neurons; although driven by either eye, binocular neurons are well modeled as gated amplifiers of their strongest monocular response, if both eyes are stimulated. Psychophysical data follows the same power law as the neural data, with a similar exponent; binocular contrast sensitivity can be modeled as a gated amplification of the more sensitive eye. These results resemble gated amplification phenomena in multisensory integration, and other non-driving modulatory interactions that affect sensory processing. Models of incomplete summation seem unnecessary for V1 facilitatory neurons or contrast sensitivity. However, binocular combination of clearly visible monocular stimuli follows Schrödinger's nonlinear magnitude-weighted average. We find that putatively suppressive binocular neurons closely follow Schrödinger's equation. Similar suppressive multisensory neurons are well documented but seldom studied. Facilitatory binocular neurons and mildly suppressive binocular neurons are likely neural correlates of binocular sensitivity and binocular appearance respectively.

Muscle activity prior to experiencing the rubber hand illusion is associated with alterations in perceived hand location.

The rubber hand illusion (RHI) is a perceptual illusion in which one is made to feel that a hand-shaped object is part of their body. This illusion is believed to be the result of the integration of afferent information. However, there has been an increasing amount of evidence that suggests efferent information plays a role in this illusion as well. Previous research has found that individuals who are afflicted by pathological lack of movement experience the RHI more vividly than control participants. Whereas individuals who move their hands more than the general population (i.e. professional pianists) experience the RHI less vividly than control participants. Based upon the available evidence it would seem that muscle activity prior to experiencing the RHI should be associated with how vividly one experiences different indices of the illusion. In the present study we tested this possibility by having participants perform a maximum voluntary muscle contraction task prior to experiencing three variants of the RHI (moving active, moving passive and classic). It was found that electromyographic features known to be indicative of muscle fatigue exhibited a positive association with proprioceptive drift when stimulation was synchronous or visual movement only (with the exception of the passive moving RHI synchronous condition). More work is needed to better characterize the muscular processes associated with experiencing the RHI.

A simple vector-like law for perceptual information combination is also followed by a class of cortical multisensory bimodal neurons.

An interdisciplinary approach to sensory information combination shows a correspondence between perceptual and neural measures of nonlinear multisensory integration. In psychophysics, sensory information combinations are often characterized by the Minkowski formula, but the neural substrates of many psychophysical multisensory interactions are unknown. We show that audiovisual interactions - for both psychophysical detection threshold data and cortical bimodal neurons - obey similar vector-like Minkowski models, suggesting that cortical bimodal neurons could underlie multisensory perceptual sensitivity. An alternative Bayesian model is not a good predictor of cortical bimodal response. In contrast to cortex, audiovisual data from superior colliculus resembles the 'City-Block' combination rule used in perceptual similarity metrics. Previous work found a simple power law amplification rule is followed for perceptual appearance measures and by cortical subthreshold multisensory neurons. The two most studied neural cell classes in cortical multisensory interactions may provide neural substrates for two important perceptual modes: appearance-based and performance-based perception.

A Simple Power Law Governs Many Sensory Amplifications and Multisensory Enhancements.

When one sensory response occurs in the presence of a different sensory stimulation, the sensory response is often amplified. The variety of sensory enhancement data tends to obscure the underlying rules, but it has long been clear that weak signals are usually amplified more than strong ones (the Principle of Inverse Effectiveness). Here we show that for many kinds of sensory amplification, the underlying law is simple and elegant: the amplified response is a power law of the unamplified response, with a compressive exponent that amplifies weak signals more than strong. For both psychophysics and cortical electrophysiology, for both humans and animals, and for both sensory integration and enhancement within a sense, gated power law amplification (amplification of one sense triggered by the presence of a different sensory signal) is often sufficient to explain sensory enhancement.

Hue opponency: chromatic valence functions, individual differences, cortical winner-take-all opponent modeling, and the relationship between spikes and sensitivity.

Neural spike rate data are more restricted in range than related psychophysical data. For example, several studies suggest a compressive (roughly cube root) nonlinear relationship between wavelength-opponent spike rates in primate midbrain and color appearance in humans, two rather widely separated domains. This presents an opportunity to partially bridge a chasm between these two domains and to probe the putative nonlinearity with other psychophysical data. Here neural wavelength-opponent data are used to create cortical competition models for hue opponency. This effort led to creation of useful models of spiking neuron winner-take-all (WTA) competition and MAX selection. When fed with actual primate data, the spiking WTA models generate reasonable wavelength-opponent spike rate behaviors. An average psychophysical observer for red-green and blue-yellow opponency is curated from eight applicable studies in the refereed and dissertation literatures, with cancellation data roughly every 10 nm in 18 subjects for yellow-blue opponency and 15 subjects for red-green opponency. A direct mapping between spiking neurons with broadband wavelength sensitivity and human psychophysical luminance yields a power law exponent of 0.27, similar to the cube root nonlinearity. Similarly, direct mapping between the WTA model opponent spike rates and psychophysical opponent data suggests power law relationships with exponents between 0.24 and 0.41.

Developmental neurotoxicity and autism: A potential link between indoor neuroactive pollutants and the curious birth order risk factor.

Epidemiological and demographic studies find an increased risk of autism among first-borns. Toxicological studies show that some semi-volatile substances found in infant products produce adverse effects in neural and endocrine systems of animals, including behavioral and developmental effects. Several factors elevate the exposure of human infants to these chemicals. The highest exposures found in infants are comparable to the exposures that induce neural toxicity in animals. A review of these literatures suggests a linking hypothesis that could bridge the epidemiological and toxicological lines of evidence: an infant's exposure to neuroactive compounds emitted by infant products is increased by product newness and abundance; exposure is likely maximized for first-born children in families that can afford new products. Exposure is reduced for subsequently-born children who reuse these now neuroactive-depleted products. The presence of neuroactive chemical emissions from infant products has implications for birth-order effects and for other curious risk factors in autism, including gender, socioeconomic status, and season-of-birth risk factors.

Bridging the divide between sensory integration and binding theory: Using a binding-like neural synchronization mechanism to model sensory enhancements during multisensory interactions.

Neural information combination problems are ubiquitous in cognitive neuroscience. Two important disciplines, although conceptually similar, take radically different approaches to these problems. Sensory binding theory is largely grounded in synchronization of neurons responding to different aspects of a stimulus, resulting in a coherent percept. Sensory integration focuses more on the influences of the senses on each other and is largely grounded in the study of neurons that respond to more than one sense. It would be desirable to bridge these disciplines, so that insights gleaned from either could be harnessed by the other. To link these two fields, we used a binding-like oscillatory synchronization mechanism to simulate neurons in rattlesnake that are driven by one sense but modulated by another. Mutual excitatory coupling produces synchronized trains of action potentials with enhanced firing rates. The same neural synchronization mechanism models the behavior of a population of cells in cat visual cortex that are modulated by auditory activation. The coupling strength of the synchronizing neurons is crucial to the outcome; a criterion of strong coupling (kept weak enough to avoid seriously distorting action potential amplitude) results in intensity-dependent sensory enhancement-the principle of inverse effectiveness-a key property of sensory integration.

Elementary visual hallucinations and their relationships to neural pattern-forming mechanisms.

An extraordinary variety of experimental (e.g., flicker, magnetic fields) and clinical (epilepsy, migraine) conditions give rise to a surprisingly common set of elementary hallucinations, including spots, geometric patterns, and jagged lines, some of which also have color, depth, motion, and texture. Many of these simple hallucinations fall into a small number of perceptual geometries-the Klüver forms-that (via a nonlinear mapping from retina to cortex) correspond to even simpler sets of oriented stripes of cortical activity (and their superpositions). Other simple hallucinations (phosphenes and fortification auras) are linked to the Klüver forms and to pattern-forming cortical mechanisms by their spatial and temporal scales. The Klüver cortical activity patterns are examples of self-organized pattern formation that arise from nonlinear dynamic interactions between excitatory and inhibitory cortical neurons; with reasonable modifications, this model accounts for a wide range of hallucinated patterns. The Klüver cortical activity patterns are a subset of autonomous spatiotemporal cortical patterns, some of which have been studied with functional imaging techniques. Understanding the interaction of these intrinsic patterns with stimulus-driven cortical activity is an important problem in neuroscience. In line with this, hallucinatory pattern formation interacts with physical stimuli, and many conditions that induce hallucinations show interesting interactions with one another. Both types of interactions are predictable from neural and psychophysical principles such as localized processing, excitatory-inhibitory neural circuits, lateral inhibition, simultaneous and sequential contrast, saccadic suppression, and perceptual opponency. Elementary hallucinations arise from familiar mechanisms stimulated in unusual ways.

To honor Fechner and obey Stevens: relationships between psychophysical and neural nonlinearities.

G. T. Fechner (1860/1966) famously described two kinds of psychophysics: Outer psychophysics captures the black box relationship between sensory inputs and perceptual magnitudes, whereas inner psychophysics contains the neural transformations that Fechner's outer psychophysics elided. The relationship between the two has never been clear. Moreover, psychophysical power laws are found in almost every sensory system, yet the vast majority of neurons show sigmoid nonlinearities. Here, we selectively review the literatures on psychophysical and physiological nonlinearities and show how they can be placed within a framework for understanding the relationship between inner and outer psychophysics: a neural organization with a logical structure commensurate to outer psychophysical theory. In theoretical treatments of Stevens's law, the power law is a consequence of combining a Weber's law scaling of inputs with a Weber's law-like scaling of sensation magnitudes, yielding an exponent that is the ratio of the Weber constants. A neural derivation using physiological sigmoid nonlinearities should be commensurate to this internal logic. There is a class of models in which two nonlinear neural mechanisms (e.g., a sensory channel and the cortical numerosity mechanism tapped by magnitude estimation) are coupled through feedback, yielding power law behavior as an emergent property of the system, with an exponent that is a ratio of neural coupling strengths. Rather than a discrepancy between psychophysics and physiology, these models suggest complementarity between inner and outer psychophysics, because the Weber constants required for outer psychophysics modeling can be derived from the sigmoid nonlinearities of inner psychophysics.

Neural interactions between flicker-induced self-organized visual hallucinations and physical stimuli.

Spontaneous pattern formation in cortical activity may have consequences for perception, but little is known about interactions between sensory-driven and self-organized cortical activity. To address this deficit, we explored the relationship between ordinary stimulus-controlled pattern perception and the autonomous hallucinatory geometrical pattern formation that occurs for unstructured visual stimulation (e.g., empty-field flicker). We found that flicker-induced hallucinations are biased by the presentation of adjacent geometrical stimuli; geometrical forms that map to cortical area V1 as orthogonal gratings are perceptually opponent in biasing hallucinations. Rotating fan blades and pulsating circular patterns are the most salient biased hallucinations. Apparent motion and fractal (1/f) noise are also effective in driving hallucinatory pattern formation (the latter is consistent with predictions of spatiotemporal pattern formation driven by stochastic resonance). The behavior of these percepts suggests that self-organized hallucinatory pattern formation in human vision is governed by the same cortical properties of localized processing, lateral inhibition, simultaneous contrast, and nonlinear retinotopic mapping that govern ordinary vision.

Fechner-Benham subjective colors do not induce McCollough after-effects.

Fechner-Benham subjective color is widely believed to be governed by local interactions in early (probably retinal) mechanisms. Here we report three lines of phenomenological evidence that suggest otherwise: subjective colors seen in spatially extended stimuli (a) are dependent on global aspects of the stimuli; (b) can become multistable in position; and (c) even after being stabilized do not support the creation of McCollough's colored after-effects--a cortically based phenomenon generally thought to be more central than Fechner-Benham color. These phenomena suggest a central locus that controls perception of subjective color, characterized by pattern dependent interactions among cortical mechanisms that draw their inputs from peripheral units.

Sensory recoding via neural synchronization: integrating hue and luminance into chromatic brightness and saturation.

If neural spike trains carry information in the frequency and timing of the spikes, then neural interactions--such as oscillatory synchronization--that alter spike frequency and timing can alter the encoded information. Using coupled oscillator theory, we show that synchronization-based processing can be used to integrate sensory information, resulting in new second-order sensory percepts signaled by the compromise frequency of the coupled system. If the signals to be coupled are nonlinearly compressed, the coupled system behaves as if it signals the product or ratio of the uncoupled signals, e.g., chromatic brightness can be signaled by the compromise frequency of coupled neurons responding to hue and luminance, and chromatic saturation can be signaled by the coupled frequency of neurons responding to hue and brightness, with a power- (Stevens's) law scaling like that observed psychophysically. These emergent properties of coupled sensory systems are intriguing because multiplicative processing and power-law scaling are fundamental aspects of sensory processing.

A role for cortical crosstalk in the binding problem: stimulus-driven correlations that link color, form, and motion.

The putative independence of cortical mechanisms for color, form, and motion raises the binding problem-how is neural activity coordinated to create unified and correctly segmented percepts? Binding could be guided by stimulus-driven correlations between mechanisms, but the nature of these correlations is largely unexplored and no one has (intentionally) studied effects on binding if this joint information is compromised. Here, we develop a theoretical framework which: (1) describes crosstalk-generated correlations between cortical mechanisms for color, achromatic form, and motion, which arise from retinogeniculate encoding; (2) shows how these correlations can facilitate synchronization, segmentation, and binding; (3) provides a basis for understanding perceptual oddities and binding failures that occur for equiluminant and stabilized images. These ideas can be tested by measuring both perceptual events and neural activity while achromatic border contrast or stabilized image velocity is manipulated.

What do catastrophic visual binding failures look like?

Ordinary vision is considered a binding success: all the pieces and aspects of an image are bound together, despite being processed by many different neurons in several different cortical areas. How this is accomplished is a key problem in visual neuroscience. The study of visual binding might be facilitated if we had ways to induce binding failures. A particularly interesting failure would involve a loss of the physical integrity of the image. Here, we identify conditions that induce such perceptual failures (e.g. the melting together of equiluminant colored images and the fragmentation of retinally stabilized images) and we suggest that these should studied using electrophysiological measures of binding.

Spatiotemporal characteristics of hemodynamic changes in the human lateral prefrontal cortex during working memory tasks.

The prefrontal cortex (PFC) is widely believed to subserve mental manipulation and monitoring processes ascribed to the central executive (CE) of working memory (WM). We attempted to examine and localize the CE by functional imaging of the frontal cortex during tasks designed to require the CE. Using near-infrared spectroscopy, we studied the spatiotemporal dynamics of oxygenated hemoglobin (oxy-Hb), an indicator of changes in regional cerebral blood flow, in both sides of lateral PFC during WM intensive tasks. In most participants, increases in oxy-Hb were localized within one subdivison during performance of the n-back task, whereas oxy-Hb increased more diffusely during the random number generation (RNG) task. Activation of the ventrolateral PFC (VLPFC) was prominent in the n-back task; both sustained and transient dynamics were observed. Transient dynamics means that oxy-Hb first increases but then decreases to less than 50% of the peak value or below the baseline level before the end of the task. For the RNG task sustained activity was also observed in the dorsolateral PFC (DLPFC), especially in the right hemisphere. However, details of patterns of activation varied across participants: subdivisions commonly activated during performance of the two tasks were the bilateral VLPFCs, either side of the VLPFC, and either side of the DLPFC in 4, 2, and 4 of the 12 participants, respectively. The remaining 2 of the 12 participants had no regions commonly activated by these tasks. These results suggest that although the PFC is implicated in the CE, there is no stereotyped anatomical PFC substrate for the CE.