Multisensory processing and proprioceptive plasticity during resizing illusions.

Bodily resizing illusions typically use visual and/or tactile inputs to produce a vivid experience of one's body changing size. Naturalistic auditory input (an input that reflects the natural sounds of a stimulus) has been used to increase illusory experience during the rubber hand illusion, whilst non-naturalistic auditory input can influence estimations of finger length. We aimed to use a non-naturalistic auditory input during a hand-based resizing illusion using augmented reality, to assess whether the addition of an auditory input would increase both subjective illusion strength and measures of performance-based tasks. Forty-four participants completed the following three conditions: no finger stretching, finger stretching without tactile feedback and finger stretching with tactile feedback. Half of the participants had an auditory input throughout all the conditions, whilst the other half did not. After each condition, the participants were given one of the following three performance tasks: stimulated (right) hand dot touch task, non-stimulated (left) hand dot touch task, and a ruler judgement task. Dot tasks involved participants reaching for the location of a virtual dot, whereas the ruler task concerned estimates of the participant's own finger on a ruler whilst the hand was hidden from view. After all trials, the participants completed a questionnaire capturing subjective illusion strength. The addition of auditory input increased subjective illusion strength for manipulations without tactile feedback but not those with tactile feedback. No facilitatory effects of audio were found for any performance task. We conclude that adding auditory input to illusory finger stretching increased subjective illusory experience in the absence of tactile feedback but did not affect performance-based measures.

A purely visual adaptation to motion can differentiate between perceptual timing and interval timing.

It is unclear whether our brain extracts and processes time information using a single-centralized mechanism or through a network of distributed mechanisms, which are specific for modality and time range. Visual adaptation has previously been used to investigate the mechanisms underlying time perception for millisecond intervals. Here, we investigated whether a well-known duration after-effect induced by motion adaptation in the sub-second range (referred to as 'perceptual timing') also occurs in the supra-second range (called 'interval timing'), which is more accessible to cognitive control. Participants judged the relative duration of two intervals after spatially localized adaptation to drifting motion. Adaptation substantially compressed the apparent duration of a 600 ms stimulus in the adapted location, whereas it had a much weaker effect on a 1200 ms interval. Discrimination thresholds after adaptation improved slightly relative to baseline, implying that the duration effect cannot be ascribed to changes in attention or to noisier estimates. A novel computational model of duration perception can explain both these results and the bidirectional shifts of perceived duration after adaptation reported in other studies. We suggest that we can use adaptation to visual motion as a tool to investigate the mechanisms underlying time perception at different time scales.

PySilSub: An open-source Python toolbox for implementing the method of silent substitution in vision and nonvisual photoreception research.

The normal human retina contains several classes of photosensitive cell-rods for low-light vision, three cone classes for daylight vision, and intrinsically photosensitive retinal ganglion cells (ipRGCs) expressing melanopsin for non-image-forming functions, including pupil control, melatonin suppression, and circadian photoentrainment. The spectral sensitivities of the photoreceptors overlap significantly, which means that most lights will stimulate all photoreceptors to varying degrees. The method of silent substitution is a powerful tool for stimulating individual photoreceptor classes selectively and has found much use in research and clinical settings. The main hardware requirement for silent substitution is a spectrally calibrated light stimulation system with at least as many primaries as there are photoreceptors under consideration. Device settings that will produce lights to selectively stimulate the photoreceptor(s) of interest can be found using a variety of analytic and algorithmic approaches. Here we present PySilSub (https://github.com/PySilentSubstitution/pysilsub), a novel Python package for silent substitution featuring flexible support for individual colorimetric observer models (including human and mouse observers), multiprimary stimulation devices, and solving silent substitution problems with linear algebra and constrained numerical optimization. The toolbox is registered with the Python Package Index and includes example data sets from various multiprimary systems. We hope that PySilSub will facilitate the application of silent substitution in research and clinical settings.

Temporal dynamics of normalization reweighting.

For decades, neural suppression in early visual cortex has been thought to be fixed. But recent work has challenged this assumption by showing that suppression can be reweighted based on recent history; when pairs of stimuli are repeatedly presented together, suppression between them strengthens. Here we investigate the temporal dynamics of this process using a steady-state visual evoked potential (SSVEP) paradigm that provides a time-resolved, direct index of suppression between pairs of stimuli flickering at different frequencies (5 and 7 Hz). Our initial analysis of an existing electroencephalography (EEG) dataset (N = 100) indicated that suppression increases substantially during the first 2-5 seconds of stimulus presentation (with some variation across stimulation frequency). We then collected new EEG data (N = 100) replicating this finding for both monocular and dichoptic mask arrangements in a preregistered study designed to measure reweighting. A third experiment (N = 20) used source-localized magnetoencephalography and found that these effects are apparent in primary visual cortex (V1), consistent with results from neurophysiological work. Because long-standing theories propose inhibition/excitation differences in autism, we also compared reweighting between individuals with high versus low autistic traits, and with and without an autism diagnosis, across our three datasets (total N = 220). We find no compelling differences in reweighting that are associated with autism. Our results support the normalization reweighting model and indicate that for prolonged stimulation, increases in suppression occur on the order of 2-5 seconds after stimulus onset.

Steady-state measures of visual suppression.

In the early visual system, suppression occurs between neurons representing different stimulus properties. This includes features such as orientation (cross-orientation suppression), eye-of-origin (interocular suppression) and spatial location (surround suppression), which are thought to involve distinct anatomical pathways. We asked if these separate routes to suppression can be differentiated by their pattern of gain control on the contrast response function measured in human participants using steady-state electroencephalography. Changes in contrast gain shift the contrast response function laterally, whereas changes in response gain scale the function vertically. We used a Bayesian hierarchical model to summarise the evidence for each type of gain control. A computational meta-analysis of 16 previous studies found the most evidence for contrast gain effects with overlaid masks, but no clear evidence favouring either response gain or contrast gain for other mask types. We then conducted two new experiments, comparing suppression from four mask types (monocular and dichoptic overlay masks, and aligned and orthogonal surround masks) on responses to sine wave grating patches flickering at 5Hz. At the occipital pole, there was strong evidence for contrast gain effects in all four mask types at the first harmonic frequency (5Hz). Suppression generally became stronger at more lateral electrode sites, but there was little evidence of response gain effects. At the second harmonic frequency (10Hz) suppression was stronger overall, and involved both contrast and response gain effects. Although suppression from different mask types involves distinct anatomical pathways, gain control processes appear to serve a common purpose, which we suggest might be to suppress less reliable inputs.

Neural markers of suppression in impaired binocular vision.

Even after conventional patching treatment, individuals with a history of amblyopia typically lack good stereo vision. This is often attributed to atypical suppression between the eyes, yet the specific mechanism is still unclear. Guided by computational models of binocular vision, we tested explicit predictions about how neural responses to contrast might differ in individuals with impaired binocular vision. Participants with a history of amblyopia (N = 25), and control participants with typical visual development (N = 19) took part in the study. Neural responses to different combinations of contrast in the left and right eyes, were measured using both electroencephalography (EEG) and functional magnetic resonance imaging (fMRI). Stimuli were sinusoidal gratings with a spatial frequency of 3c/deg, flickering at 4 Hz. In the fMRI experiment, we also ran population receptive field and retinotopic mapping sequences, and a phase-encoded localiser stimulus, to identify voxels in primary visual cortex (V1) sensitive to the main stimulus. Neural responses in both modalities increased monotonically with stimulus contrast. When measured with EEG, responses were attenuated in the weaker eye, consistent with a fixed tonic suppression of that eye. When measured with fMRI, a low contrast stimulus in the weaker eye substantially reduced the response to a high contrast stimulus in the stronger eye. This effect was stronger than when the stimulus-eye pairings were reversed, consistent with unbalanced dynamic suppression between the eyes. Measuring neural responses using different methods leads to different conclusions about visual differences in individuals with impaired binocular vision. Both of the atypical suppression effects may relate to binocular perceptual deficits, e.g. in stereopsis, and we anticipate that these measures could be informative for monitoring the progress of treatments aimed at recovering binocular vision.

Dynamic properties of internal noise probed by modulating binocular rivalry.

Neural systems are inherently noisy, and this noise can affect our perception from moment to moment. This is particularly apparent in binocular rivalry, where perception of competing stimuli shown to the left and right eyes alternates over time. We modulated rivalling stimuli using dynamic sequences of external noise of various rates and amplitudes. We repeated each external noise sequence twice, and assessed the consistency of percepts across repetitions. External noise modulations of sufficiently high contrast increased consistency scores above baseline, and were most effective at 1/8Hz. A computational model of rivalry in which internal noise has a 1/f (pink) temporal amplitude spectrum, and a standard deviation of 16% contrast, provided the best account of our data. Our novel technique provides detailed estimates of the dynamic properties of internal noise during binocular rivalry, and by extension the stochastic processes that drive our perception and other types of spontaneous brain activity.

The Effect of Locomotion on Early Visual Contrast Processing in Humans.

Most of our knowledge about vision comes from experiments in which stimuli are presented to immobile human subjects or animals. In the case of human subjects, movement during psychophysical, electrophysiological, or neuroimaging experiments is considered to be a source of noise to be eliminated. Animals used in visual neuroscience experiments are typically restrained and, in many cases, anesthetized. In reality, however, vision is often used to guide the motion of awake, ambulating organisms. Recent work in mice has shown that locomotion elevates visual neuronal response amplitudes (Niell and Stryker, 2010; Erisken et al., 2014; Fu et al., 2014; Lee et al., 2014; Mineault et al., 2016) and reduces long-range gain control (Ayaz et al., 2013). Here, we used both psychophysics and steady-state electrophysiology to investigate whether similar effects of locomotion on early visual processing can be measured in humans. Our psychophysical results show that brisk walking has little effect on subjects' ability to detect briefly presented contrast changes and that co-oriented flankers are, if anything, more effective masks when subjects are walking. Our electrophysiological data were consistent with the psychophysics indicating no increase in stimulus-driven neuronal responses while walking and no reduction in surround suppression. In summary, we have found evidence that early contrast processing is altered by locomotion in humans but in a manner that differs from that reported in mice. The effects of locomotion on very low-level visual processing may differ on a species-by-species basis and may reflect important differences in the levels of arousal associated with locomotion.SIGNIFICANCE STATEMENT Mice are the current model of choice for studying low-level visual processing. Recent studies have shown that mouse visual cortex is modulated by behavioral state: primary visual cortex neurons in locomoting mice tend to be more sensitive and less influenced by long-range gain control. Here, we tested these effects in humans by measuring psychophysical detection thresholds and electroencephalography (EEG) responses while subjects walked on a treadmill. We found no evidence of increased contrast sensitivity or reduced surround suppression in walking humans. Our data show that fundamental measurements of early visual processing differ between humans and mice and this has important implications for recent work on the links among arousal, behavior, and vision in these two species.

Internal noise in contrast discrimination propagates forwards from early visual cortex.

Human contrast discrimination performance is limited by transduction nonlinearities and variability of the neural representation (noise). Whereas the nonlinearities have been well-characterised, there is less agreement about the specifics of internal noise. Psychophysical models assume that it impacts late in sensory processing, whereas neuroimaging and intracranial electrophysiology studies suggest that the noise is much earlier. We investigated whether perceptually-relevant internal noise arises in early visual areas or later decision making areas. We recorded EEG and MEG during a two-interval-forced-choice contrast discrimination task and used multivariate pattern analysis to decode target/non-target and selected/non-selected intervals from evoked responses. We found that perceptual decisions could be decoded from both EEG and MEG signals, even when the stimuli in both intervals were physically identical. Above-chance decision classification started <100 ms after stimulus onset, suggesting that neural noise affects sensory signals early in the visual pathway. Classification accuracy increased over time, peaking at >500 ms. Applying multivariate analysis to separate anatomically-defined brain regions in MEG source space, we found that occipital regions were informative early on but then information spreads forwards across parietal and frontal regions. This is consistent with neural noise affecting sensory processing at multiple stages of perceptual decision making. We suggest how early sensory noise might be resolved with Birdsall's linearisation, in which a dominant noise source obscures subsequent nonlinearities, to allow the visual system to preserve the wide dynamic range of early areas whilst still benefitting from contrast-invariance at later stages. A preprint of this work is available at: https://doi.org/10.1101/364612.

Selectivity for mid-level properties of faces and places in the fusiform face area and parahippocampal place area.

Regions in the ventral visual pathway, such as the fusiform face area (FFA) and parahippocampal place area (PPA) are selective for images from specific object categories. Yet images from different object categories differ in their image properties. To investigate how these image properties are represented in the FFA and PPA, we compared neural responses to locally-SCRAMBLED images (in which mid-level, spatial properties are preserved) and globally-SCRAMBLED images (in which mid-level, spatial properties are not preserved). There was a greater response in the FFA and PPA to images from the preferred CATEGORY relative to their non-preferred category for the scrambled conditions. However, there was a greater selectivity for locally-scrambled compared to globally-scrambled images. Next, we compared the magnitude of fMR-adaptation to intact and scrambled images. fMR-adaptation was evident to locally-scrambled images from the preferred category. However, there was no adaptation to globally-scrambled images from the preferred category. These results show that the selectivity to faces and places in the FFA and PPA is dependent on mid-level properties of the image that are preserved by local-scrambling.

A data-driven approach to stimulus selection reveals an image-based representation of objects in high-level visual areas.

The ventral visual pathway is directly involved in the perception and recognition of objects. However, the extent to which the neural representation of objects in this region reflects low-level or high-level properties remains unresolved. A problem in resolving this issue is that only a small proportion of the objects experienced during natural viewing can be shown during a typical experiment. This can lead to an uneven sampling of objects that biases our understanding of how they are represented. To address this issue, we developed a data-driven approach to stimulus selection that involved describing a large number objects in terms of their image properties. In the first experiment, clusters of objects were evenly selected from this multi-dimensional image space. Although the clusters did not have any consistent semantic features, each elicited a distinct pattern of neural response. In the second experiment, we asked whether high-level, category-selective patterns of response could be elicited by objects from other categories, but with similar image properties. Object clusters were selected based on the similarity of their image properties to objects from five different categories (bottle, chair, face, house, and shoe). The pattern of response to each metameric object cluster was similar to the pattern elicited by objects from the corresponding category. For example, the pattern for bottles was similar to the pattern for objects with similar image properties to bottles. In both experiments, the patterns of response were consistent across participants providing evidence for common organising principles. This study provides a more ecological approach to understanding the perceptual representations of objects and reveals the importance of image properties.

Supplemental Vitamin B-12 Enhances the Neural Response to Sensory Stimulation in the Barrel Cortex of Healthy Rats but Does Not Affect Spontaneous Neural Activity.

BACKGROUND: Although vitamin B-12 (B-12) is known to contribute to the structural and functional development of the brain, it is unclear if B-12 supplementation has any beneficial effect in healthy populations in terms of enhanced neurologic status of the brain or improved cognitive function. OBJECTIVES: We investigated the effect of dietary supplementation of B-12 on the cortical neural activity of well-nourished young adult rats and tested the hypothesis that B-12 supplementation in healthy rats may reduce sensory-evoked neural activity due to enhanced inhibition. METHODS: Female Lister Hooded rats weighing 190-265 g (2-4 mo old) were included in the study. The experimental group was fed with B-12 (cyanocobalamin)-enriched water at a concentration of 1 mg/L, and the control (CON) group with tap water for 3 wk. Animals were then anesthetized and cortical neural responses to whisker stimulation were recorded in vivo through the use of a multichannel microelectrode, from which local field potentials (LFPs) were extracted. RESULTS: Somatosensory-evoked LFP was 25% larger in the B-12 group (4.13 ± 0.24 mV) than in the CON group (3.30 ± 0.21 mV) (P = 0.02). Spontaneous neural activity did not differ between groups; frequency spectra at each frequency bin of interest did not pass the cluster-forming threshold at the 5% significance level. CONCLUSIONS: These findings do not provide evidence supporting the hypothesis of decreased neural activity due to B-12 supplementation. As the spontaneous neural activity was unaffected, the increase in somatosensory-evoked LFP may be due to enhanced afferent signal reaching the barrel cortex from the whisker pad, indicating that B-12-supplemented rats may have enhanced sensitivity to sensory stimulation compared with the CON group. We suggest that this enhancement might be the result of lowered sensory threshold, although the underlying mechanism has yet to be elucidated.

Binocular vision adaptively suppresses delayed monocular signals.

A neutral density filter placed before one eye will produce a dichoptic imbalance in luminance, which attenuates responses to visual stimuli and lags neural signals from retina to cortex in the filtered eye. When stimuli are presented to both the filtered and unfiltered eye (i.e., binocularly), neural responses show little attenuation and no lag compared with their baseline counterpart. This suggests that binocular visual mechanisms must suppress the attenuated and delayed input from the filtered eye; however, the mechanisms involved remain unclear. Here, we used a Steady-State Visual Evoked Potential (SSVEP) technique to measure neural responses to monocularly and binocularly presented stimuli while observers wore an ND filter in front of their dominant eye. These data were well-described by a binocular summation model, which received the sinusoidal contrast modulation of the stimulus as input. We incorporated the influence of the ND filter with an impulse response function, which adjusted the input magnitude and phase in a biophysically plausible manner. The model captured the increase in attenuation and lag of neural signals for stimuli presented to the filtered eye as a function of filter strength, while also generating the filter phase-invariant responses from binocular presentation for EEG and psychophysical data. These results clarify how binocular visual mechanisms-specifically interocular suppression-can suppress the delayed and attenuated signals from the filtered eye and maintain normal neural signals under imbalanced luminance conditions.

Autism sensory dysfunction in an evolutionarily conserved system.

There is increasing evidence for a strong genetic basis for autism, with many genetic models being developed in an attempt to replicate autistic symptoms in animals. However, current animal behaviour paradigms rarely match the social and cognitive behaviours exhibited by autistic individuals. Here, we instead assay another functional domain-sensory processing-known to be affected in autism to test a novel genetic autism model in Drosophila melanogaster. We show similar visual response alterations and a similar development trajectory in Nhe3 mutant flies (total n = 72) and in autistic human participants (total n = 154). We report a dissociation between first- and second-order electrophysiological visual responses to steady-state stimulation in adult mutant fruit flies that is strikingly similar to the response pattern in human adults with ASD as well as that of a large sample of neurotypical individuals with high numbers of autistic traits. We explain this as a genetically driven, selective signalling alteration in transient visual dynamics. In contrast to adults, autistic children show a decrease in the first-order response that is matched by the fruit fly model, suggesting that a compensatory change in processing occurs during development. Our results provide the first animal model of autism comprising a differential developmental phenotype in visual processing.

Evidence for an Optimal Algorithm Underlying Signal Combination in Human Visual Cortex.

How does the cortex combine information from multiple sources? We tested several computational models against data from steady-state electroencephalography (EEG) experiments in humans, using periodic visual stimuli combined across either retinal location or eye-of-presentation. A model in which signals are raised to an exponent before being summed in both the numerator and the denominator of a gain control nonlinearity gave the best account of the data. This model also predicted the pattern of responses in a range of additional conditions accurately and with no free parameters, as well as predicting responses at harmonic and intermodulation frequencies between 1 and 30 Hz. We speculate that this model implements the optimal algorithm for combining multiple noisy inputs, in which responses are proportional to the weighted sum of both inputs. This suggests a novel purpose for cortical gain control: implementing optimal signal combination via mutual inhibition, perhaps explaining its ubiquity as a neural computation.

Autistic Traits in the Neurotypical Population do not Predict Increased Response Conservativeness in Perceptual Decision Making.

Recent research has shown that adults and children with autism spectrum disorders have a more conservative decision criterion in perceptual decision making compared to neurotypical individuals, meaning that autistic participants prioritise accuracy over speed of a decision. Here, we test whether autistic traits in the neurotypical population correlate with increased response conservativeness. We employed three different tasks; for two tasks we recruited participants from China ( N = 39) and for one task from the United Kingdom ( N = 37). Our results show that autistic traits in the neurotypical population do not predict variation in response criterion. We also failed to replicate previous work showing a relationship between autistic traits and sensitivity to coherent motion and static orientation. Following the argument proposed by Gregory and Plaisted-Grant, we discuss why perceptual differences between autistic and neurotypical participants do not necessarily predict perceptual differences between neurotypical participants with high and low autistic traits.

Decoding eye-of-origin outside of awareness.

In the primary visual cortex of many mammals, ocular dominance columns segregate information from the two eyes. Yet under controlled conditions, most human observers are unable to correctly report the eye to which a stimulus has been shown, indicating that this information is lost during subsequent processing. This study investigates whether eye-of-origin information is available in the pattern of electrophysiological activity evoked by visual stimuli, recorded using EEG and decoded using multivariate pattern analysis. Observers (N=24) viewed sine-wave grating and plaid stimuli of different orientations, shown to either the left or right eye (or both). Using a support vector machine, eye-of-origin could be decoded above chance at around 140 and 220ms post stimulus onset, yet observers were at chance for reporting this information. Other stimulus features, such as binocularity, orientation, spatial pattern, and the presence of interocular conflict (i.e. rivalry), could also be decoded using the same techniques, though all of these were perceptually discriminable above chance. A control analysis found no evidence to support the possibility that eye dominance was responsible for the eye-of-origin effects. These results support a structural explanation for multivariate decoding of electrophysiological signals - information organised in cortical columns can be decoded, even when observers are unaware of this information.

Measuring nonlinear signal combination using EEG.

Relatively little is known about the processes, both linear and nonlinear, by which signals are combined beyond V1. By presenting two stimulus components simultaneously, flickering at different temporal frequencies (frequency tagging) while measuring steady-state visual evoked potentials, we can assess responses to the individual components, including direct measurements of suppression on each other, and various nonlinear responses to their combination found at intermodulation frequencies. The result is a rather rich dataset of frequencies at which responses can be found. We presented pairs of sinusoidal gratings at different temporal frequencies, forming plaid patterns that were "coherent" (looking like a checkerboard) and "noncoherent" (looking like a pair of transparently overlaid gratings), and found clear intermodulation responses to compound stimuli, indicating nonlinear summation. This might have been attributed to cross-orientation suppression except that the pattern of intermodulation responses differed for coherent and noncoherent patterns, whereas the effects of suppression (measured at the component frequencies) did not. A two-stage model of nonlinear summation involving conjunction detection with a logical AND gate described the data well, capturing the difference between coherent and noncoherent plaids over a wide array of possible response frequencies. Multistimulus frequency-tagged EEG in combination with computational modeling may be a very valuable tool in studying the conjunction of these signals. In the current study the results suggest a second-order mechanism responding selectively to coherent plaid patterns.

Category-selective patterns of neural response in the ventral visual pathway in the absence of categorical information.

Neuroimaging studies have revealed distinct patterns of response to different object categories in the ventral visual pathway. These findings imply that object category is an important organizing principle in this region of visual cortex. However, object categories also differ systematically in their image properties. So, it is possible that these patterns of neural response could reflect differences in image properties rather than object category. To differentiate between these alternative explanations, we used images of objects that had been phase-scrambled at a local or global level. Both scrambling processes preserved many of the lower-level image properties, but rendered the images unrecognizable. We then measured the effect of image scrambling on the patterns of neural response within the ventral pathway. We found that intact and scrambled images evoked distinct category-selective patterns of activity in the ventral stream. Moreover, intact and scrambled images of the same object category produced highly similar patterns of response. These results suggest that the neural representation in the ventral visual pathway is tightly linked to the statistical properties of the image.

Measurement of crosstalk in stereoscopic display systems used for vision research.

Studying binocular vision requires precise control over the stimuli presented to the left and right eyes. A popular technique is to segregate signals either temporally (frame interleaving), spectrally (using colored filters), or through light polarization. None of these segregation methods achieves perfect isolation, and so a degree of crosstalk is usually apparent, in which signals intended for one eye are faintly visible to the other eye. Previous studies have reported crosstalk values mostly for consumer-grade systems. Here we measure crosstalk for eight systems, many of which are intended for use in vision research. We provide benchmark crosstalk values, report a negative crosstalk effect in some LCD-based systems, and give guidelines for dealing with crosstalk in different experimental paradigms.