Multisensory processing and proprioceptive plasticity during resizing illusions.

Bodily resizing illusions typically use visual and/or tactile inputs to produce a vivid experience of one's body changing size. Naturalistic auditory input (an input that reflects the natural sounds of a stimulus) has been used to increase illusory experience during the rubber hand illusion, whilst non-naturalistic auditory input can influence estimations of finger length. We aimed to use a non-naturalistic auditory input during a hand-based resizing illusion using augmented reality, to assess whether the addition of an auditory input would increase both subjective illusion strength and measures of performance-based tasks. Forty-four participants completed the following three conditions: no finger stretching, finger stretching without tactile feedback and finger stretching with tactile feedback. Half of the participants had an auditory input throughout all the conditions, whilst the other half did not. After each condition, the participants were given one of the following three performance tasks: stimulated (right) hand dot touch task, non-stimulated (left) hand dot touch task, and a ruler judgement task. Dot tasks involved participants reaching for the location of a virtual dot, whereas the ruler task concerned estimates of the participant's own finger on a ruler whilst the hand was hidden from view. After all trials, the participants completed a questionnaire capturing subjective illusion strength. The addition of auditory input increased subjective illusion strength for manipulations without tactile feedback but not those with tactile feedback. No facilitatory effects of audio were found for any performance task. We conclude that adding auditory input to illusory finger stretching increased subjective illusory experience in the absence of tactile feedback but did not affect performance-based measures.

Temporal dynamics of normalization reweighting.

For decades, neural suppression in early visual cortex has been thought to be fixed. But recent work has challenged this assumption by showing that suppression can be reweighted based on recent history; when pairs of stimuli are repeatedly presented together, suppression between them strengthens. Here we investigate the temporal dynamics of this process using a steady-state visual evoked potential (SSVEP) paradigm that provides a time-resolved, direct index of suppression between pairs of stimuli flickering at different frequencies (5 and 7 Hz). Our initial analysis of an existing electroencephalography (EEG) dataset (N = 100) indicated that suppression increases substantially during the first 2-5 seconds of stimulus presentation (with some variation across stimulation frequency). We then collected new EEG data (N = 100) replicating this finding for both monocular and dichoptic mask arrangements in a preregistered study designed to measure reweighting. A third experiment (N = 20) used source-localized magnetoencephalography and found that these effects are apparent in primary visual cortex (V1), consistent with results from neurophysiological work. Because long-standing theories propose inhibition/excitation differences in autism, we also compared reweighting between individuals with high versus low autistic traits, and with and without an autism diagnosis, across our three datasets (total N = 220). We find no compelling differences in reweighting that are associated with autism. Our results support the normalization reweighting model and indicate that for prolonged stimulation, increases in suppression occur on the order of 2-5 seconds after stimulus onset.

Different rules for binocular combination of luminance flicker in cortical and subcortical pathways.

How does the human brain combine information across the eyes? It has been known for many years that cortical normalization mechanisms implement 'ocularity invariance': equalizing neural responses to spatial patterns presented either monocularly or binocularly. Here, we used a novel combination of electrophysiology, psychophysics, pupillometry, and computational modeling to ask whether this invariance also holds for flickering luminance stimuli with no spatial contrast. We find dramatic violations of ocularity invariance for these stimuli, both in the cortex and also in the subcortical pathways that govern pupil diameter. Specifically, we find substantial binocular facilitation in both pathways with the effect being strongest in the cortex. Near-linear binocular additivity (instead of ocularity invariance) was also found using a perceptual luminance matching task. Ocularity invariance is, therefore, not a ubiquitous feature of visual processing, and the brain appears to repurpose a generic normalization algorithm for different visual functions by adjusting the amount of interocular suppression.

Enhancing precision in human neuroscience.

Human neuroscience has always been pushing the boundary of what is measurable. During the last decade, concerns about statistical power and replicability - in science in general, but also specifically in human neuroscience - have fueled an extensive debate. One important insight from this discourse is the need for larger samples, which naturally increases statistical power. An alternative is to increase the precision of measurements, which is the focus of this review. This option is often overlooked, even though statistical power benefits from increasing precision as much as from increasing sample size. Nonetheless, precision has always been at the heart of good scientific practice in human neuroscience, with researchers relying on lab traditions or rules of thumb to ensure sufficient precision for their studies. In this review, we encourage a more systematic approach to precision. We start by introducing measurement precision and its importance for well-powered studies in human neuroscience. Then, determinants for precision in a range of neuroscientific methods (MRI, M/EEG, EDA, Eye-Tracking, and Endocrinology) are elaborated. We end by discussing how a more systematic evaluation of precision and the application of respective insights can lead to an increase in reproducibility in human neuroscience.

PySilSub: An open-source Python toolbox for implementing the method of silent substitution in vision and nonvisual photoreception research.

The normal human retina contains several classes of photosensitive cell-rods for low-light vision, three cone classes for daylight vision, and intrinsically photosensitive retinal ganglion cells (ipRGCs) expressing melanopsin for non-image-forming functions, including pupil control, melatonin suppression, and circadian photoentrainment. The spectral sensitivities of the photoreceptors overlap significantly, which means that most lights will stimulate all photoreceptors to varying degrees. The method of silent substitution is a powerful tool for stimulating individual photoreceptor classes selectively and has found much use in research and clinical settings. The main hardware requirement for silent substitution is a spectrally calibrated light stimulation system with at least as many primaries as there are photoreceptors under consideration. Device settings that will produce lights to selectively stimulate the photoreceptor(s) of interest can be found using a variety of analytic and algorithmic approaches. Here we present PySilSub (https://github.com/PySilentSubstitution/pysilsub), a novel Python package for silent substitution featuring flexible support for individual colorimetric observer models (including human and mouse observers), multiprimary stimulation devices, and solving silent substitution problems with linear algebra and constrained numerical optimization. The toolbox is registered with the Python Package Index and includes example data sets from various multiprimary systems. We hope that PySilSub will facilitate the application of silent substitution in research and clinical settings.

A purely visual adaptation to motion can differentiate between perceptual timing and interval timing.

It is unclear whether our brain extracts and processes time information using a single-centralized mechanism or through a network of distributed mechanisms, which are specific for modality and time range. Visual adaptation has previously been used to investigate the mechanisms underlying time perception for millisecond intervals. Here, we investigated whether a well-known duration after-effect induced by motion adaptation in the sub-second range (referred to as 'perceptual timing') also occurs in the supra-second range (called 'interval timing'), which is more accessible to cognitive control. Participants judged the relative duration of two intervals after spatially localized adaptation to drifting motion. Adaptation substantially compressed the apparent duration of a 600 ms stimulus in the adapted location, whereas it had a much weaker effect on a 1200 ms interval. Discrimination thresholds after adaptation improved slightly relative to baseline, implying that the duration effect cannot be ascribed to changes in attention or to noisier estimates. A novel computational model of duration perception can explain both these results and the bidirectional shifts of perceived duration after adaptation reported in other studies. We suggest that we can use adaptation to visual motion as a tool to investigate the mechanisms underlying time perception at different time scales.

Distinct neural signatures of multimodal resizing illusions.

Illusory body resizing typically uses multisensory integration to change the perceived size of a body part. Previous studies associate these multisensory body illusions with frontal theta oscillations and parietal gamma oscillations for dis-integration and integration of multisensory signals, respectively. However, recent studies also support illusory changes of embodiment from unimodal visual stimuli. This preregistered study (N = 48) investigated differences between multisensory visuo-tactile and unimodal visual resizing illusions using EEG, to gain a more comprehensive understanding of the neural underpinnings of resizing illusions in a healthy population. We hypothesised (1) stronger illusion in multisensory compared to unimodal, and unimodal compared to incongruent (dis-integration) conditions, (2) greater parietal gamma during multisensory compared to unimodal, and (3) greater frontal theta during incongruent compared to baseline conditions. Subjective Illusory results partially support Hypothesis 1, showing a stronger illusion in multisensory compared to unimodal conditions, but finding no significant difference comparing unimodal to incongruent conditions. Results partially supported EEG hypotheses, finding increased parietal gamma activity comparing multisensory to unimodal visual conditions, happening at a later stage of the illusion when compared to previous rubber hand illusion EEG findings, whilst also finding increased parietal theta activity when comparing incongruent to non-illusion conditions. While results demonstrated that only 27% of participants experienced the stretching illusion with unimodal visual stimuli compared to 73% of participants experiencing the stretching illusion in the multisensory condition, further analysis suggested that those who experience visual-only illusions exhibit a different neural signature to those who do not, with activity focussed around frontal and parietal regions early on in the illusory manipulation, compared to activity focussed more over parietal regions and at a later point in the illusory manipulation for the full sample of participants. Our results replicate previous subjective experience findings and support the importance of multisensory integration for illusory changes in perceived body size, whilst adding to our understanding of the temporal onset of multisensory integration within resizing illusions, differing from that of rubber hand illusions.

Blurring the boundary between models and reality: Visual perception of scale assessed by performance.

One of the primary jobs of visual perception is to build a three-dimensional representation of the world around us from our flat retinal images. These are a rich source of depth cues but no single one of them can tell us about scale (i.e., absolute depth and size). For example, the pictorial depth cues in a (perfect) scale model are identical to those in the real scene that is being modelled. Here we investigate image blur gradients, which derive naturally from the limited depth of field available for any optical device and can be used to help estimate visual scale. By manipulating image blur artificially to produce what is sometimes called fake tilt shift miniaturization, we provide the first performance-based evidence that human vision uses this cue when making forced-choice judgements about scale (identifying which of an image pair was a photograph of a full-scale railway scene, and which was a 1:76 scale model). The orientation of the blur gradient (relative to the ground plane) proves to be crucial, though its rate of change is less important for our task, suggesting a fairly coarse visual analysis of this image parameter.

Object Image Size Is a Fundamental Coding Dimension in Human Vision: New Insights and Model.

In previous psychophysical work we found that luminance contrast is integrated over retinal area subject to contrast gain control. If different mechanisms perform this operation for a range of superimposed retinal regions of different sizes, this could provide the basis for size-coding. To test this idea we included two novel features in a standard adaptation paradigm to discount more pedestrian accounts of repulsive size-aftereffects. First, we used spatially jittering luminance-contrast adaptors to avoid simple contour displacement aftereffects. Second, we decoupled adaptor and target spatial frequency to avoid the well-known spatial frequency shift aftereffect. Empirical results indicated strong evidence of a bidirectional size adaptation aftereffect. We show that the textbook population model is inappropriate for our results, and develop our existing model of contrast perception to include multiple size mechanisms with divisive surround-suppression from the largest mechanism. For a given stimulus patch, this delivers a blurred step-function of responses across the population, with contrast and size encoded by the height and lateral position of the step. Unlike for textbook population coding schemes, our human results (N = 4 male, N = 4 female) displayed two asymmetries: (i) size aftereffects were greatest for targets smaller than the adaptor, and (ii) on that side of the function, results did not return to baseline, even when targets were 25% of adaptor diameter. Our results and emergent model properties provide evidence for a novel dimension of visual coding (size) and a novel strategy for that coding, consistent with previous results on contrast detection and discrimination for various stimulus sizes.

Neural Responses to Novel and Existing Words in Children with Autism Spectrum and Developmental Language Disorder.

The formation of new phonological representations is key in establishing items in the mental lexicon. Phonological forms become stable with repetition, time and sleep. Atypicality in the establishment of new word forms is characteristic of children with developmental language disorder (DLD) and autism spectrum disorder (ASD), yet neural changes in response to novel word forms over time have not yet been directly compared in these groups. This study measured habituation of event-related-potentials (ERPs) to novel and known words within and between two sessions spaced 24 hours apart in typically developing (TD) children, and their peers with DLD or ASD. We hypothesised that modulation of the auditory N400 amplitude would mark real-time changes in lexical processing with habituation evident within and across sessions in the TD group, while the DLD group would show attenuated habituation within sessions, and the ASD group attenuated habituation between sessions. Twenty-one typically developing children, 19 children with ASD, and 16 children with DLD listened passively to known and novel words on two consecutive days, while ERPs were recorded using dry electrodes. Counter to our hypotheses, no habituation effect emerged within sessions. However, responses did habituate between sessions, with this effect being reduced in the DLD group, indicating less pre-activation of lexical representations in response to words encountered the previous day. No differences in change over time were observed between the TD and ASD groups. These data are in keeping with theories stressing the importance of sleep-related consolidation in word learning.

A psychophysical performance-based approach to the quality assessment of image processing algorithms.

Image processing algorithms are used to improve digital image representations in either their appearance or storage efficiency. The merit of these algorithms depends, in part, on visual perception by human observers. However, in practice, most are assessed numerically, and the perceptual metrics that do exist are criterion sensitive with several shortcomings. Here we propose an objective performance-based perceptual measure of image quality and demonstrate this by comparing the efficacy of a denoising algorithm for a variety of filters. For baseline, we measured detection thresholds for a white noise signal added to one of a pair of natural images in a two-alternative forced-choice (2AFC) paradigm where each image was selected randomly from a set of n = 308 on each trial. In a series of experimental conditions, the stimulus image pairs were passed through various configurations of a denoising algorithm. The differences in noise detection thresholds with and without denoising are objective perceptual measures of the ability of the algorithm to render noise invisible. This was a factor of two (6dB) in our experiment and consistent across a range of filter bandwidths and types. We also found that thresholds in all conditions converged on a common value of PSNR, offering support for this metric. We discuss how the 2AFC approach might be used for other algorithms including compression, deblurring and edge-detection. Finally, we provide a derivation for our Cartesian-separable log-Gabor filters, with polar parameters. For the biological vision community this has some advantages over the more typical (i) polar-separable variety and (ii) Cartesian-separable variety with Cartesian parameters.

Steady-state measures of visual suppression.

In the early visual system, suppression occurs between neurons representing different stimulus properties. This includes features such as orientation (cross-orientation suppression), eye-of-origin (interocular suppression) and spatial location (surround suppression), which are thought to involve distinct anatomical pathways. We asked if these separate routes to suppression can be differentiated by their pattern of gain control on the contrast response function measured in human participants using steady-state electroencephalography. Changes in contrast gain shift the contrast response function laterally, whereas changes in response gain scale the function vertically. We used a Bayesian hierarchical model to summarise the evidence for each type of gain control. A computational meta-analysis of 16 previous studies found the most evidence for contrast gain effects with overlaid masks, but no clear evidence favouring either response gain or contrast gain for other mask types. We then conducted two new experiments, comparing suppression from four mask types (monocular and dichoptic overlay masks, and aligned and orthogonal surround masks) on responses to sine wave grating patches flickering at 5Hz. At the occipital pole, there was strong evidence for contrast gain effects in all four mask types at the first harmonic frequency (5Hz). Suppression generally became stronger at more lateral electrode sites, but there was little evidence of response gain effects. At the second harmonic frequency (10Hz) suppression was stronger overall, and involved both contrast and response gain effects. Although suppression from different mask types involves distinct anatomical pathways, gain control processes appear to serve a common purpose, which we suggest might be to suppress less reliable inputs.

Neural markers of suppression in impaired binocular vision.

Even after conventional patching treatment, individuals with a history of amblyopia typically lack good stereo vision. This is often attributed to atypical suppression between the eyes, yet the specific mechanism is still unclear. Guided by computational models of binocular vision, we tested explicit predictions about how neural responses to contrast might differ in individuals with impaired binocular vision. Participants with a history of amblyopia (N = 25), and control participants with typical visual development (N = 19) took part in the study. Neural responses to different combinations of contrast in the left and right eyes, were measured using both electroencephalography (EEG) and functional magnetic resonance imaging (fMRI). Stimuli were sinusoidal gratings with a spatial frequency of 3c/deg, flickering at 4 Hz. In the fMRI experiment, we also ran population receptive field and retinotopic mapping sequences, and a phase-encoded localiser stimulus, to identify voxels in primary visual cortex (V1) sensitive to the main stimulus. Neural responses in both modalities increased monotonically with stimulus contrast. When measured with EEG, responses were attenuated in the weaker eye, consistent with a fixed tonic suppression of that eye. When measured with fMRI, a low contrast stimulus in the weaker eye substantially reduced the response to a high contrast stimulus in the stronger eye. This effect was stronger than when the stimulus-eye pairings were reversed, consistent with unbalanced dynamic suppression between the eyes. Measuring neural responses using different methods leads to different conclusions about visual differences in individuals with impaired binocular vision. Both of the atypical suppression effects may relate to binocular perceptual deficits, e.g. in stereopsis, and we anticipate that these measures could be informative for monitoring the progress of treatments aimed at recovering binocular vision.

Power contours: Optimising sample size and precision in experimental psychology and human neuroscience.

When designing experimental studies with human participants, experimenters must decide how many trials each participant will complete, as well as how many participants to test. Most discussion of statistical power (the ability of a study design to detect an effect) has focused on sample size, and assumed sufficient trials. Here we explore the influence of both factors on statistical power, represented as a 2-dimensional plot on which iso-power contours can be visualized. We demonstrate the conditions under which the number of trials is particularly important, that is, when the within-participant variance is large relative to the between-participants variance. We then derive power contour plots using existing data sets for 8 experimental paradigms and methodologies (including reaction times, sensory thresholds, fMRI, MEG, and EEG), and provide example code to calculate estimates of the within- and between-participants variance for each method. In all cases, the within-participant variance was larger than the between-participants variance, meaning that the number of trials has a meaningful influence on statistical power in commonly used paradigms. An online tool is provided (https://shiny.york.ac.uk/powercontours/) for generating power contours, from which the optimal combination of trials and participants can be calculated when designing future studies. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Nonlinear transduction of emotional facial expression.

To create neural representations of external stimuli, the brain performs a number of processing steps that transform its inputs. For fundamental attributes, such as stimulus contrast, this involves one or more nonlinearities that are believed to optimise the neural code to represent features of the natural environment. Here we ask if the same is also true of more complex stimulus dimensions, such as emotional facial expression. We report the results of three experiments combining morphed facial stimuli with electrophysiological and psychophysical methods to measure the function mapping emotional expression intensity to internal response. The results converge on a nonlinearity that accelerates over weak expressions, and then becomes shallower for stronger expressions, similar to the situation for lower level stimulus properties. We further demonstrate that the nonlinearity is not attributable to the morphing procedure used in stimulus generation.

A data-driven approach to stimulus selection reveals an image-based representation of objects in high-level visual areas.

The ventral visual pathway is directly involved in the perception and recognition of objects. However, the extent to which the neural representation of objects in this region reflects low-level or high-level properties remains unresolved. A problem in resolving this issue is that only a small proportion of the objects experienced during natural viewing can be shown during a typical experiment. This can lead to an uneven sampling of objects that biases our understanding of how they are represented. To address this issue, we developed a data-driven approach to stimulus selection that involved describing a large number objects in terms of their image properties. In the first experiment, clusters of objects were evenly selected from this multi-dimensional image space. Although the clusters did not have any consistent semantic features, each elicited a distinct pattern of neural response. In the second experiment, we asked whether high-level, category-selective patterns of response could be elicited by objects from other categories, but with similar image properties. Object clusters were selected based on the similarity of their image properties to objects from five different categories (bottle, chair, face, house, and shoe). The pattern of response to each metameric object cluster was similar to the pattern elicited by objects from the corresponding category. For example, the pattern for bottles was similar to the pattern for objects with similar image properties to bottles. In both experiments, the patterns of response were consistent across participants providing evidence for common organising principles. This study provides a more ecological approach to understanding the perceptual representations of objects and reveals the importance of image properties.

Dynamic properties of internal noise probed by modulating binocular rivalry.

Neural systems are inherently noisy, and this noise can affect our perception from moment to moment. This is particularly apparent in binocular rivalry, where perception of competing stimuli shown to the left and right eyes alternates over time. We modulated rivalling stimuli using dynamic sequences of external noise of various rates and amplitudes. We repeated each external noise sequence twice, and assessed the consistency of percepts across repetitions. External noise modulations of sufficiently high contrast increased consistency scores above baseline, and were most effective at 1/8Hz. A computational model of rivalry in which internal noise has a 1/f (pink) temporal amplitude spectrum, and a standard deviation of 16% contrast, provided the best account of our data. Our novel technique provides detailed estimates of the dynamic properties of internal noise during binocular rivalry, and by extension the stochastic processes that drive our perception and other types of spontaneous brain activity.

Supplemental Vitamin B-12 Enhances the Neural Response to Sensory Stimulation in the Barrel Cortex of Healthy Rats but Does Not Affect Spontaneous Neural Activity.

BACKGROUND: Although vitamin B-12 (B-12) is known to contribute to the structural and functional development of the brain, it is unclear if B-12 supplementation has any beneficial effect in healthy populations in terms of enhanced neurologic status of the brain or improved cognitive function. OBJECTIVES: We investigated the effect of dietary supplementation of B-12 on the cortical neural activity of well-nourished young adult rats and tested the hypothesis that B-12 supplementation in healthy rats may reduce sensory-evoked neural activity due to enhanced inhibition. METHODS: Female Lister Hooded rats weighing 190-265 g (2-4 mo old) were included in the study. The experimental group was fed with B-12 (cyanocobalamin)-enriched water at a concentration of 1 mg/L, and the control (CON) group with tap water for 3 wk. Animals were then anesthetized and cortical neural responses to whisker stimulation were recorded in vivo through the use of a multichannel microelectrode, from which local field potentials (LFPs) were extracted. RESULTS: Somatosensory-evoked LFP was 25% larger in the B-12 group (4.13 ± 0.24 mV) than in the CON group (3.30 ± 0.21 mV) (P = 0.02). Spontaneous neural activity did not differ between groups; frequency spectra at each frequency bin of interest did not pass the cluster-forming threshold at the 5% significance level. CONCLUSIONS: These findings do not provide evidence supporting the hypothesis of decreased neural activity due to B-12 supplementation. As the spontaneous neural activity was unaffected, the increase in somatosensory-evoked LFP may be due to enhanced afferent signal reaching the barrel cortex from the whisker pad, indicating that B-12-supplemented rats may have enhanced sensitivity to sensory stimulation compared with the CON group. We suggest that this enhancement might be the result of lowered sensory threshold, although the underlying mechanism has yet to be elucidated.

Internal noise in contrast discrimination propagates forwards from early visual cortex.

Human contrast discrimination performance is limited by transduction nonlinearities and variability of the neural representation (noise). Whereas the nonlinearities have been well-characterised, there is less agreement about the specifics of internal noise. Psychophysical models assume that it impacts late in sensory processing, whereas neuroimaging and intracranial electrophysiology studies suggest that the noise is much earlier. We investigated whether perceptually-relevant internal noise arises in early visual areas or later decision making areas. We recorded EEG and MEG during a two-interval-forced-choice contrast discrimination task and used multivariate pattern analysis to decode target/non-target and selected/non-selected intervals from evoked responses. We found that perceptual decisions could be decoded from both EEG and MEG signals, even when the stimuli in both intervals were physically identical. Above-chance decision classification started <100 ms after stimulus onset, suggesting that neural noise affects sensory signals early in the visual pathway. Classification accuracy increased over time, peaking at >500 ms. Applying multivariate analysis to separate anatomically-defined brain regions in MEG source space, we found that occipital regions were informative early on but then information spreads forwards across parietal and frontal regions. This is consistent with neural noise affecting sensory processing at multiple stages of perceptual decision making. We suggest how early sensory noise might be resolved with Birdsall's linearisation, in which a dominant noise source obscures subsequent nonlinearities, to allow the visual system to preserve the wide dynamic range of early areas whilst still benefitting from contrast-invariance at later stages. A preprint of this work is available at: https://doi.org/10.1101/364612.

Selectivity for mid-level properties of faces and places in the fusiform face area and parahippocampal place area.

Regions in the ventral visual pathway, such as the fusiform face area (FFA) and parahippocampal place area (PPA) are selective for images from specific object categories. Yet images from different object categories differ in their image properties. To investigate how these image properties are represented in the FFA and PPA, we compared neural responses to locally-SCRAMBLED images (in which mid-level, spatial properties are preserved) and globally-SCRAMBLED images (in which mid-level, spatial properties are not preserved). There was a greater response in the FFA and PPA to images from the preferred CATEGORY relative to their non-preferred category for the scrambled conditions. However, there was a greater selectivity for locally-scrambled compared to globally-scrambled images. Next, we compared the magnitude of fMR-adaptation to intact and scrambled images. fMR-adaptation was evident to locally-scrambled images from the preferred category. However, there was no adaptation to globally-scrambled images from the preferred category. These results show that the selectivity to faces and places in the FFA and PPA is dependent on mid-level properties of the image that are preserved by local-scrambling.