Decoding Reach Direction in Early "Visual" Cortex of Congenitally Blind Individuals.

Motor actions, such as reaching or grasping, can be decoded from fMRI activity of early visual cortex (EVC) in sighted humans. This effect can depend on vision or visual imagery, or alternatively, could be driven by mechanisms independent of visual experience. Here, we show that the actions of reaching in different directions can be reliably decoded from fMRI activity of EVC in congenitally blind humans (both sexes). Thus, neither visual experience nor visual imagery is necessary for EVC to represent action-related information. We also demonstrate that, within EVC of blind humans, the accuracy of reach direction decoding is highest in areas typically representing foveal vision and gradually decreases in areas typically representing peripheral vision. We propose that this might indicate the existence of a predictive, hard-wired mechanism of aligning action and visual spaces. This mechanism might send action-related information primarily to the high-resolution foveal visual areas, which are critical for guiding and online correction of motor actions. Finally, we show that, beyond EVC, the decoding of reach direction in blind humans is most accurate in dorsal stream areas known to be critical for visuo-spatial and visuo-motor integration in the sighted. Thus, these areas can develop space and action representations even in the lifelong absence of vision. Overall, our findings in congenitally blind humans match previous research on the action system in the sighted, and suggest that the development of action representations in the human brain might be largely independent of visual experience.SIGNIFICANCE STATEMENT Early visual cortex (EVC) was traditionally thought to process only visual signals from the retina. Recent studies proved this account incomplete, and showed EVC involvement in many activities not directly related to incoming visual information, such as memory, sound, or action processing. Is EVC involved in these activities because of visual imagery? Here, we show robust reach direction representation in EVC of humans born blind. This demonstrates that EVC can represent actions independently of vision and visual imagery. Beyond EVC, we found that reach direction representation in blind humans is strongest in dorsal brain areas, critical for action processing in the sighted. This suggests that the development of action representations in the human brain is largely independent of visual experience.

Why cognitive penetration of our perceptual experience is still the most plausible account.

To what extent is our perceptual experience influenced by higher cognitive phenomena like beliefs, desires, concepts, templates? Given recent arguments against the possibility of cognitive penetration, we present striking evidence against the impenetrability claims. The weak impenetrability claim cannot account for (1) extensive structural feedback organization of the brain, (2) temporally very early feedback loops and (3) functional top-down processes modulating early visual processes by category-specific information. The strong impenetrability claim could incorporate these data by widening the "perceptual module" such that it includes rich but still internal processing in a very large perceptual module. We argue that this latter view leads to an implausible version of a module. Therefore, we have to accept cognitive penetration of our perceptual experience as the best theoretical account so far given the available empirical evidence. We outline that this does not have any problematic consequences for the relation between perception and cognition.

TMS over V5 disrupts motion prediction.

Given the vast amount of sensory information the brain has to deal with, predicting some of this information based on the current context is a resource-efficient strategy. The framework of predictive coding states that higher-level brain areas generate a predictive model to be communicated via feedback connections to early sensory areas. Here, we directly tested the necessity of a higher-level visual area, V5, in this predictive processing in the context of an apparent motion paradigm. We flashed targets on the apparent motion trace in-time or out-of-time with the predicted illusory motion token. As in previous studies, we found that predictable in-time targets were better detected than unpredictable out-of-time targets. However, when we applied functional magnetic resonance imaging-guided, double-pulse transcranial magnetic stimulation (TMS) over left V5 at 13-53 ms before target onset, the detection advantage of in-time targets was eliminated; this was not the case when TMS was applied over the vertex. Our results are causal evidence that V5 is necessary for a prediction effect, which has been shown to modulate V1 activity (Alink et al. 2010). Thus, our findings suggest that information processing between V5 and V1 is crucial for visual motion prediction, providing experimental support for the predictive coding framework.

Varieties of cognitive penetration in visual perception.

Is our perceptual experience a veridical representation of the world or is it a product of our beliefs and past experiences? Cognitive penetration describes the influence of higher level cognitive factors on perceptual experience and has been a debated topic in philosophy of mind and cognitive science. Here, we focus on visual perception, particularly early vision, and how it is affected by contextual expectations and memorized cognitive contents. We argue for cognitive penetration based on recent empirical evidence demonstrating contextual and top-down influences on early visual processes. On the basis of a perceptual model, we propose different types of cognitive penetration depending on the processing level on which the penetration happens and depending on where the penetrating influence comes from. Our proposal has two consequences: (1) the traditional controversy on whether cognitive penetration occurs or not is ill posed, and (2) a clear-cut perception-cognition boundary cannot be maintained.

How the eyes respond to sounds.

Eye movements have been extensively studied with respect to visual stimulation. However, we live in a multisensory world, and how the eyes are driven by other senses has been explored much less. Here, we review the evidence on how audition can trigger and drive different eye responses and which cortical and subcortical neural correlates are involved. We provide an overview on how different types of sounds, from simple tones and noise bursts to spatially localized sounds and complex linguistic stimuli, influence saccades, microsaccades, smooth pursuit, pupil dilation, and eye blinks. The reviewed evidence reveals how the auditory system interacts with the oculomotor system, both behaviorally and neurally, and how this differs from visually driven eye responses. Some evidence points to multisensory interaction, and potential multisensory integration, but the underlying computational and neural mechanisms are still unclear. While there are marked differences in how the eyes respond to auditory compared to visual stimuli, many aspects of auditory-evoked eye responses remain underexplored, and we summarize the key open questions for future research.

Impaired presynaptic function and elimination of synapses at premature stages during postnatal development of the cerebellum in the absence of CALEB (CSPG5/neuroglycan C).

Chicken acidic leucine-rich EGF-like domain-containing brain protein (CALEB), also known as chondroitin sulfate proteoglycan (CSPG)5 or neuroglycan C, is a neural chondroitin sulfate-containing and epidermal growth factor (EGF)-domain-containing transmembrane protein that is implicated in synaptic maturation. Here, we studied the role of CALEB within the developing cerebellum. Adult CALEB-deficient mice displayed impaired motor coordination in Rota-Rod experiments. Analysis of the neuronal connectivity of Purkinje cells by patch-clamp recordings demonstrated impairments of presynaptic maturation of inhibitory synapses. GABAergic synapses on Purkinje cells revealed decreased evoked amplitudes, altered paired-pulse facilitation and reduced depression after repetitive stimulation at early postnatal but not at mature stages. Furthermore, the elimination of supernumerary climbing fiber synapses on Purkinje cells was found to occur at earlier developmental stages in the absence of CALEB. For example, at postnatal day 8 in wild-type mice, 54% of Purkinje cells had three or more climbing fiber synapses in contrast to mutants where this number was decreased to less than 25%. The basic properties of the climbing fiber Purkinje cell synapse remained unaffected. Using Sholl analysis of dye-injected Purkinje cells we revealed that the branching pattern of the dendritic tree of Purkinje cells was not impaired in CALEB-deficient mice. The alterations observed by patch-clamp recordings correlated with a specific pattern and timing of expression of CALEB in Purkinje cells, i.e. it is dynamically regulated during development from a high chondroitin sulfate-containing form to a non-chondroitin sulfate-containing form. Thus, our results demonstrated an involvement of CALEB in the presynaptic differentiation of cerebellar GABAergic synapses and revealed a new role for CALEB in synapse elimination in Purkinje cells.

A candidate for the attentional bottleneck: set-size specific modulation of the right TPJ during attentive enumeration.

Several recent behavioral studies have shown that the enumeration of a small number of items (a process termed subitizing) depends on the availability of attentional resources and is not a preattentive process as previously thought. Here we studied the neural correlates of visual enumeration under different attentional loads in a dual-task paradigm using fMRI. Relatively intact subitizing under low attentional load compared to impaired subitizing under high attentional load was associated with an increase in BOLD signal in the right temporo-parietal junction (rTPJ). Crucially, attentionally modulated response in the rTPJ was specific to small set sizes (up to 3 items) and did not occur at larger set sizes (5-7 items). This result has two implications: (1) Subitizing involves part of the fronto-parietal network for stimulus-driven attention providing neural evidence against preattentive subitizing. (2) Activity in rTPJ is set-size modulated. Together with similar evidence from studies probing visual short-term memory, this result suggests that rTPJ modulation might reflect the brain's ability to attentively handle small set sizes. Thus, the rTPJ may play an important role for the emergence of a capacity limit in both enumeration and visual short-term memory.

Unconscious numerical priming despite interocular suppression.

Whether high-level properties of stimuli rendered invisible by interocular competition can influence perception and behavior remains controversial. We studied whether suppressed and invisible symbolic and nonsymbolic numerical stimuli can elicit priming. First, we established that participants were objectively unable to discriminate numerical prime stimuli when interocular suppression rendered them invisible. Next, we asked participants to enumerate a visible target set of items after being exposed to a suppressed, invisible (nonsymbolic or symbolic) prime set. Both symbolic and nonsymbolic unconsciously perceived numerical primes induced robust priming effects that were specific to the numerical distance between the target and prime. Comparison with a no-prime condition revealed that primes larger than targets interfered with target enumeration and primes the same as or smaller than targets facilitated target enumeration. Taken together, our findings provide clear evidence for high-level processing of stimuli rendered invisible through interocular suppression.

Decoding Natural Sounds in Early "Visual" Cortex of Congenitally Blind Individuals.

Complex natural sounds, such as bird singing, people talking, or traffic noise, induce decodable fMRI activation patterns in early visual cortex of sighted blindfolded participants [1]. That is, early visual cortex receives non-visual and potentially predictive information from audition. However, it is unclear whether the transfer of auditory information to early visual areas is an epiphenomenon of visual imagery or, alternatively, whether it is driven by mechanisms independent from visual experience. Here, we show that we can decode natural sounds from activity patterns in early "visual" areas of congenitally blind individuals who lack visual imagery. Thus, visual imagery is not a prerequisite of auditory feedback to early visual cortex. Furthermore, the spatial pattern of sound decoding accuracy in early visual cortex was remarkably similar in blind and sighted individuals, with an increasing decoding accuracy gradient from foveal to peripheral regions. This suggests that the typical organization by eccentricity of early visual cortex develops for auditory feedback, even in the lifelong absence of vision. The same feedback to early visual cortex might support visual perception in the sighted [1] and drive the recruitment of this area for non-visual functions in blind individuals [2, 3].

Emotional faces guide the eyes in the absence of awareness.

The ability to act quickly to a threat is a key skill for survival. Under awareness, threat-related emotional information, such as an angry or fearful face, has not only perceptual advantages but also guides rapid actions such as eye movements. Emotional information that is suppressed from awareness still confers perceptual and attentional benefits. However, it is unknown whether suppressed emotional information can directly guide actions, or whether emotional information has to enter awareness to do so. We suppressed emotional faces from awareness using continuous flash suppression and tracked eye gaze position. Under successful suppression, as indicated by objective and subjective measures, gaze moved towards fearful faces, but away from angry faces. Our findings reveal that: (1) threat-related emotional stimuli can guide eye movements in the absence of visual awareness; (2) threat-related emotional face information guides distinct oculomotor actions depending on the type of threat conveyed by the emotional expression.

Predictive feedback to V1 dynamically updates with sensory input.

Predictive coding theories propose that the brain creates internal models of the environment to predict upcoming sensory input. Hierarchical predictive coding models of vision postulate that higher visual areas generate predictions of sensory inputs and feed them back to early visual cortex. In V1, sensory inputs that do not match the predictions lead to amplified brain activation, but does this amplification process dynamically update to new retinotopic locations with eye-movements? We investigated the effect of eye-movements in predictive feedback using functional brain imaging and eye-tracking whilst presenting an apparent motion illusion. Apparent motion induces an internal model of motion, during which sensory predictions of the illusory motion feed back to V1. We observed attenuated BOLD responses to predicted stimuli at the new post-saccadic location in V1. Therefore, pre-saccadic predictions update their retinotopic location in time for post-saccadic input, validating dynamic predictive coding theories in V1.

Decoding sound and imagery content in early visual cortex.

Human early visual cortex was traditionally thought to process simple visual features such as orientation, contrast, and spatial frequency via feedforward input from the lateral geniculate nucleus (e.g., [1]). However, the role of nonretinal influence on early visual cortex is so far insufficiently investigated despite much evidence that feedback connections greatly outnumber feedforward connections [2-5]. Here, we explored in five fMRI experiments how information originating from audition and imagery affects the brain activity patterns in early visual cortex in the absence of any feedforward visual stimulation. We show that category-specific information from both complex natural sounds and imagery can be read out from early visual cortex activity in blindfolded participants. The coding of nonretinal information in the activity patterns of early visual cortex is common across actual auditory perception and imagery and may be mediated by higher-level multisensory areas. Furthermore, this coding is robust to mild manipulations of attention and working memory but affected by orthogonal, cognitively demanding visuospatial processing. Crucially, the information fed down to early visual cortex is category specific and generalizes to sound exemplars of the same category, providing evidence for abstract information feedback rather than precise pictorial feedback. Our results suggest that early visual cortex receives nonretinal input from other brain areas when it is generated by auditory perception and/or imagery, and this input carries common abstract information. Our findings are compatible with feedback of predictive information to the earliest visual input level (e.g., [6]), in line with predictive coding models [7-10].

Dissociation of prediction from conscious perception.

The framework of predictive coding offers a parsimonious explanation for many perceptual phenomena. According to this framework, perception of the outer world is created by the comparison of incoming sensory information with an internal predictive model based on previous experience and context. However, it is unclear whether the predicted percept needs to enter conscious awareness for the internal predictive model to be effective. Here we used an apparent motion paradigm to show that while prediction and conscious awareness of a predicted percept may coincide, a dissociation can be observed. When sensory information provides reliable input for the internal predictive model, the predicted percept does not have to be consciously perceived for successful prediction. However, when sensory input is ambiguous, conscious awareness helps the prediction to take effect.

Transfer of predictive signals across saccades.

Predicting visual information facilitates efficient processing of visual signals. Higher visual areas can support the processing of incoming visual information by generating predictive models that are fed back to lower visual areas. Functional brain imaging has previously shown that predictions interact with visual input already at the level of the primary visual cortex (V1; Harrison et al., 2007; Alink et al., 2010). Given that fixation changes up to four times a second in natural viewing conditions, cortical predictions are effective in V1 only if they are fed back in time for the processing of the next stimulus and at the corresponding new retinotopic position. Here, we tested whether spatio-temporal predictions are updated before, during, or shortly after an inter-hemifield saccade is executed, and thus, whether the predictive signal is transferred swiftly across hemifields. Using an apparent motion illusion, we induced an internal motion model that is known to produce a spatio-temporal prediction signal along the apparent motion trace in V1 (Muckli et al., 2005; Alink et al., 2010). We presented participants with both visually predictable and unpredictable targets on the apparent motion trace. During the task, participants saccaded across the illusion whilst detecting the target. As found previously, predictable stimuli were detected more frequently than unpredictable stimuli. Furthermore, we found that the detection advantage of predictable targets is detectable as early as 50-100 ms after saccade offset. This result demonstrates the rapid nature of the transfer of a spatio-temporally precise predictive signal across hemifields, in a paradigm previously shown to modulate V1.

EEG activation differences in the pre-motor cortex and supplementary motor area between normal individuals with high and low traits of autism.

The human mirror neuron system (hMNS) is believed to provide a basic mechanism for social cognition. Event-related desynchronization (ERD) in alpha (8-12Hz) and low beta band (12-20Hz) over sensori-motor cortex has been suggested to index mirror neurons' activity. We tested whether autistic traits revealed by high and low scores on the Autistic Quotient (AQ) in the normal population are linked to variations in the electroencephalogram (EEG) over motor, pre-motor cortex and supplementary motor area (SMA) during action observation. Results revealed that in the low AQ group, the pre-motor cortex and SMA were more active during hand action than static hand observation whereas in the high AQ group the same areas were active both during static and hand action observation. In fact participants with high traits of autism showed greater low beta ERD while observing the static hand than those with low traits and this low beta ERD was not significantly different when they watched hand actions. Over primary motor cortex, the classical alpha and low beta ERD during hand actions relative to static hand observation was found across all participants. These findings suggest that the observation-execution matching system works differently according to the degree of autism traits in the normal population and that this is differentiated in terms of the EEG according to scalp site and bandwidth.

Reduced cortico-motor facilitation in a normal sample with high traits of autism.

Recent research in social neuroscience proposes a link between mirror neuron system (MNS) and social cognition. The MNS has been proposed to be the neural mechanism underlying action recognition and intention understanding and more broadly social cognition. Pre-motor MNS has been suggested to modulate the motor cortex during action observation. This modulation results in an enhanced cortico-motor excitability reflected in increased motor evoked potentials (MEPs) at the muscle of interest during action observation. Anomalous MNS activity has been reported in the autistic population whose social skills are notably impaired. It is still an open question whether traits of autism in the normal population are linked to the MNS functioning. We measured TMS-induced MEPs in normal individuals with high and low traits of autism as measured by the autistic quotient (AQ), while observing videos of hand or mouth actions, static images of a hand or mouth or a blank screen. No differences were observed between the two while they observed a blank screen. However participants with low traits of autism showed significantly greater MEP amplitudes during observation of hand/mouth actions relative to static hand/mouth stimuli. In contrast, participants with high traits of autism did not show such a MEP amplitude difference between observation of actions and static stimuli. These results are discussed with reference to MNS functioning.

Modulating attentional load affects numerosity estimation: evidence against a pre-attentive subitizing mechanism.

Traditionally, the visual enumeration of a small number of items (1 to about 4), referred to as subitizing, has been thought of as a parallel and pre-attentive process and functionally different from the serial attentive enumeration of larger numerosities. We tested this hypothesis by employing a dual task paradigm that systematically manipulated the attentional resources available to an enumeration task. Enumeration accuracy for small numerosities was severely decreased as more attentional resources were taken away from the numerical task, challenging the traditionally held notion of subitizing as a pre-attentive, capacity-independent process. Judgement of larger numerosities was also affected by dual task conditions and attentional load. These results challenge the proposal that small numerosities are enumerated by a mechanism separate from large numerosities and support the idea of a single, attention-demanding enumeration mechanism.