Split-Brain: What We Know Now and Why This is Important for Understanding Consciousness.

Recently, the discussion regarding the consequences of cutting the corpus callosum ("split-brain") has regained momentum (Corballis, Corballis, Berlucchi, & Marzi, Brain, 141(6), e46, 2018; Pinto et al., Brain, 140(5), 1231-1237, 2017a; Pinto, Lamme, & de Haan, Brain, 140(11), e68, 2017; Volz & Gazzaniga, Brain, 140(7), 2051-2060, 2017; Volz, Hillyard, Miller, & Gazzaniga, Brain, 141(3), e15, 2018). This collective review paper aims to summarize the empirical common ground, to delineate the different interpretations, and to identify the remaining questions. In short, callosotomy leads to a broad breakdown of functional integration ranging from perception to attention. However, the breakdown is not absolute as several processes, such as action control, seem to remain unified. Disagreement exists about the responsible mechanisms for this remaining unity. The main issue concerns the first-person perspective of a split-brain patient. Does a split-brain harbor a split consciousness or is consciousness unified? The current consensus is that the body of evidence is insufficient to answer this question, and different suggestions are made with respect to how future studies might address this paucity. In addition, it is suggested that the answers might not be a simple yes or no but that intermediate conceptualizations need to be considered.

Scene complexity modulates degree of feedback activity during object detection in natural scenes.

Selective brain responses to objects arise within a few hundreds of milliseconds of neural processing, suggesting that visual object recognition is mediated by rapid feed-forward activations. Yet disruption of neural responses in early visual cortex beyond feed-forward processing stages affects object recognition performance. Here, we unite these discrepant findings by reporting that object recognition involves enhanced feedback activity (recurrent processing within early visual cortex) when target objects are embedded in natural scenes that are characterized by high complexity. Human participants performed an animal target detection task on natural scenes with low, medium or high complexity as determined by a computational model of low-level contrast statistics. Three converging lines of evidence indicate that feedback was selectively enhanced for high complexity scenes. First, functional magnetic resonance imaging (fMRI) activity in early visual cortex (V1) was enhanced for target objects in scenes with high, but not low or medium complexity. Second, event-related potentials (ERPs) evoked by target objects were selectively enhanced at feedback stages of visual processing (from ~220 ms onwards) for high complexity scenes only. Third, behavioral performance for high complexity scenes deteriorated when participants were pressed for time and thus less able to incorporate the feedback activity. Modeling of the reaction time distributions using drift diffusion revealed that object information accumulated more slowly for high complexity scenes, with evidence accumulation being coupled to trial-to-trial variation in the EEG feedback response. Together, these results suggest that while feed-forward activity may suffice to recognize isolated objects, the brain employs recurrent processing more adaptively in naturalistic settings, using minimal feedback for simple scenes and increasing feedback for complex scenes.

Split brain: divided perception but undivided consciousness.

In extensive studies with two split-brain patients we replicate the standard finding that stimuli cannot be compared across visual half-fields, indicating that each hemisphere processes information independently of the other. Yet, crucially, we show that the canonical textbook findings that a split-brain patient can only respond to stimuli in the left visual half-field with the left hand, and to stimuli in the right visual half-field with the right hand and verbally, are not universally true. Across a wide variety of tasks, split-brain patients with a complete and radiologically confirmed transection of the corpus callosum showed full awareness of presence, and well above chance-level recognition of location, orientation and identity of stimuli throughout the entire visual field, irrespective of response type (left hand, right hand, or verbally). Crucially, we used confidence ratings to assess conscious awareness. This revealed that also on high confidence trials, indicative of conscious perception, response type did not affect performance. These findings suggest that severing the cortical connections between hemispheres splits visual perception, but does not create two independent conscious perceivers within one brain.

NMDA Receptor Antagonist Ketamine Distorts Object Recognition by Reducing Feedback to Early Visual Cortex.

It is a well-established fact that top-down processes influence neural representations in lower-level visual areas. Electrophysiological recordings in monkeys as well as theoretical models suggest that these top-down processes depend on NMDA receptor functioning. However, this underlying neural mechanism has not been tested in humans. We used fMRI multivoxel pattern analysis to compare the neural representations of ambiguous Mooney images before and after they were recognized with their unambiguous grayscale version. Additionally, we administered ketamine, an NMDA receptor antagonist, to interfere with this process. Our results demonstrate that after recognition, the pattern of brain activation elicited by a Mooney image is more similar to that of its easily recognizable grayscale version than to the pattern evoked by the identical Mooney image before recognition. Moreover, recognition of Mooney images decreased mean response; however, neural representations of separate images became more dissimilar. So from the neural perspective, unrecognizable Mooney images all "look the same", whereas recognized Mooneys look different. We observed these effects in posterior fusiform part of lateral occipital cortex and in early visual cortex. Ketamine distorted these effects of recognition, but in early visual cortex only. This suggests that top-down processes from higher- to lower-level visual areas might operate via an NMDA pathway.

The time course of natural scene perception with reduced attention.

Attention is thought to impose an informational bottleneck on vision by selecting particular information from visual scenes for enhanced processing. Behavioral evidence suggests, however, that some scene information is extracted even when attention is directed elsewhere. Here, we investigated the neural correlates of this ability by examining how attention affects electrophysiological markers of scene perception. In two electro-encephalography (EEG) experiments, human subjects categorized real-world scenes as manmade or natural (full attention condition) or performed tasks on unrelated stimuli in the center or periphery of the scenes (reduced attention conditions). Scene processing was examined in two ways: traditional trial averaging was used to assess the presence of a categorical manmade/natural distinction in event-related potentials, whereas single-trial analyses assessed whether EEG activity was modulated by scene statistics that are diagnostic of naturalness of individual scenes. The results indicated that evoked activity up to 250 ms was unaffected by reduced attention, showing intact categorical differences between manmade and natural scenes and strong modulations of single-trial activity by scene statistics in all conditions. Thus initial processing of both categorical and individual scene information remained intact with reduced attention. Importantly, however, attention did have profound effects on later evoked activity; full attention on the scene resulted in prolonged manmade/natural differences, increased neural sensitivity to scene statistics, and enhanced scene memory. These results show that initial processing of real-world scene information is intact with diminished attention but that the depth of processing of this information does depend on attention.

Neural Correlates of Visual Short-term Memory Dissociate between Fragile and Working Memory Representations.

Evidence is accumulating that the classic two-stage model of visual STM (VSTM), comprising iconic memory (IM) and visual working memory (WM), is incomplete. A third memory stage, termed fragile VSTM (FM), seems to exist in between IM and WM [Vandenbroucke, A. R. E., Sligte, I. G., & Lamme, V. A. F. Manipulations of attention dissociate fragile visual STM from visual working memory. Neuropsychologia, 49, 1559-1568, 2011; Sligte, I. G., Scholte, H. S., & Lamme, V. A. F. Are there multiple visual STM stores? PLoS One, 3, e1699, 2008]. Although FM can be distinguished from IM using behavioral and fMRI methods, the question remains whether FM is a weak expression of WM or a separate form of memory with its own neural signature. Here, we tested whether FM and WM in humans are supported by dissociable time-frequency features of EEG recordings. Participants performed a partial-report change detection task, from which individual differences in FM and WM capacity were estimated. These individual FM and WM capacities were correlated with time-frequency characteristics of the EEG signal before and during encoding and maintenance of the memory display. FM capacity showed negative alpha correlations over peri-occipital electrodes, whereas WM capacity was positively related, suggesting increased visual processing (lower alpha) to be related to FM capacity. Furthermore, FM capacity correlated with an increase in theta power over central electrodes during preparation and processing of the memory display, whereas WM did not. In addition to a difference in visual processing characteristics, a positive relation between gamma power and FM capacity was observed during both preparation and maintenance periods of the task. On the other hand, we observed that theta-gamma coupling was negatively correlated with FM capacity, whereas it was slightly positively correlated with WM. These data show clear differences in the neural substrates of FM versus WM and suggest that FM depends more on visual processing mechanisms compared with WM. This study thus provides novel evidence for a dissociation between different stages in VSTM.

Top-down modulation in human visual cortex predicts the stability of a perceptual illusion.

Conscious perception sometimes fluctuates strongly, even when the sensory input is constant. For example, in motion-induced blindness (MIB), a salient visual target surrounded by a moving pattern suddenly disappears from perception, only to reappear after some variable time. Whereas such changes of perception result from fluctuations of neural activity, mounting evidence suggests that the perceptual changes, in turn, may also cause modulations of activity in several brain areas, including visual cortex. In this study, we asked whether these latter modulations might affect the subsequent dynamics of perception. We used magnetoencephalography (MEG) to measure modulations in cortical population activity during MIB. We observed a transient, retinotopically widespread modulation of beta (12-30 Hz)-frequency power over visual cortex that was closely linked to the time of subjects' behavioral report of the target disappearance. This beta modulation was a top-down signal, decoupled from both the physical stimulus properties and the motor response but contingent on the behavioral relevance of the perceptual change. Critically, the modulation amplitude predicted the duration of the subsequent target disappearance. We propose that the transformation of the perceptual change into a report triggers a top-down mechanism that stabilizes the newly selected perceptual interpretation.

Pupil size tracks perceptual content and surprise.

Changes in pupil size at constant light levels reflect the activity of neuromodulatory brainstem centers that control global brain state. These endogenously driven pupil dynamics can be synchronized with cognitive acts. For example, the pupil dilates during the spontaneous switches of perception of a constant sensory input in bistable perceptual illusions. It is unknown whether this pupil dilation only indicates the occurrence of perceptual switches, or also their content. Here, we measured pupil diameter in human subjects reporting the subjective disappearance and re-appearance of a physically constant visual target surrounded by a moving pattern ('motion-induced blindness' illusion). We show that the pupil dilates during the perceptual switches in the illusion and a stimulus-evoked 'replay' of that illusion. Critically, the switch-related pupil dilation encodes perceptual content, with larger amplitude for disappearance than re-appearance. This difference in pupil response amplitude enables prediction of the type of report (disappearance vs. re-appearance) on individual switches (receiver-operating characteristic: 61%). The amplitude difference is independent of the relative durations of target-visible and target-invisible intervals and subjects' overt behavioral report of the perceptual switches. Further, we show that pupil dilation during the replay also scales with the level of surprise about the timing of switches, but there is no evidence for an interaction between the effects of surprise and perceptual content on the pupil response. Taken together, our results suggest that pupil-linked brain systems track both the content of, and surprise about, perceptual events.

Neuronal integration in visual cortex elevates face category tuning to conscious face perception.

The human brain has the extraordinary capability to transform cluttered sensory input into distinct object representations. For example, it is able to rapidly and seemingly without effort detect object categories in complex natural scenes. Surprisingly, category tuning is not sufficient to achieve conscious recognition of objects. What neural process beyond category extraction might elevate neural representations to the level where objects are consciously perceived? Here we show that visible and invisible faces produce similar category-selective responses in the ventral visual cortex. The pattern of neural activity evoked by visible faces could be used to decode the presence of invisible faces and vice versa. However, only visible faces caused extensive response enhancements and changes in neural oscillatory synchronization, as well as increased functional connectivity between higher and lower visual areas. We conclude that conscious face perception is more tightly linked to neural processes of sustained information integration and binding than to processes accommodating face category tuning.

Expectations accelerate entry of visual stimuli into awareness.

How do expectations influence transitions between unconscious and conscious perceptual processing? According to the influential predictive processing framework, perceptual content is determined by predictive models of the causes of sensory signals. On one interpretation, conscious contents arise when predictive models are verified by matching sensory input (minimizing prediction error). On another, conscious contents arise when surprising events falsify current perceptual predictions. Finally, the cognitive impenetrability account posits that conscious perception is not affected by such higher level factors. To discriminate these positions, we combined predictive cueing with continuous flash suppression (CFS) in which the relative contrast of a target image gradually increases over time. In four experiments we established that expected stimuli enter consciousness faster than neutral or unexpected stimuli. These effects are difficult to account for in terms of response priming, pre-existing stimulus associations, or the attentional mechanisms that cause asynchronous temporal order judgments (of simultaneously presented stimuli). Our results further suggest that top-down expectations play a larger role when bottom-up input is ambiguous, in line with predictive processing accounts of perception. Taken together, our findings support the hypothesis that conscious access depends on verification of perceptual predictions.

From image statistics to scene gist: evoked neural activity reveals transition from low-level natural image structure to scene category.

The visual system processes natural scenes in a split second. Part of this process is the extraction of "gist," a global first impression. It is unclear, however, how the human visual system computes this information. Here, we show that, when human observers categorize global information in real-world scenes, the brain exhibits strong sensitivity to low-level summary statistics. Subjects rated a specific instance of a global scene property, naturalness, for a large set of natural scenes while EEG was recorded. For each individual scene, we derived two physiologically plausible summary statistics by spatially pooling local contrast filter outputs: contrast energy (CE), indexing contrast strength, and spatial coherence (SC), indexing scene fragmentation. We show that behavioral performance is directly related to these statistics, with naturalness rating being influenced in particular by SC. At the neural level, both statistics parametrically modulated single-trial event-related potential amplitudes during an early, transient window (100-150 ms), but SC continued to influence activity levels later in time (up to 250 ms). In addition, the magnitude of neural activity that discriminated between man-made versus natural ratings of individual trials was related to SC, but not CE. These results suggest that global scene information may be computed by spatial pooling of responses from early visual areas (e.g., LGN or V1). The increased sensitivity over time to SC in particular, which reflects scene fragmentation, suggests that this statistic is actively exploited to estimate scene naturalness.

Opposing dorsal/ventral stream dynamics during figure-ground segregation.

The visual system has been commonly subdivided into two segregated visual processing streams: The dorsal pathway processes mainly spatial information, and the ventral pathway specializes in object perception. Recent findings, however, indicate that different forms of interaction (cross-talk) exist between the dorsal and the ventral stream. Here, we used TMS and concurrent EEG recordings to explore these interactions between the dorsal and ventral stream during figure-ground segregation. In two separate experiments, we used repetitive TMS and single-pulse TMS to disrupt processing in the dorsal (V5/HMT⁺) and the ventral (lateral occipital area) stream during a motion-defined figure discrimination task. We presented stimuli that made it possible to differentiate between relatively low-level (figure boundary detection) from higher-level (surface segregation) processing steps during figure-ground segregation. Results show that disruption of V5/HMT⁺ impaired performance related to surface segregation; this effect was mainly found when V5/HMT⁺ was perturbed in an early time window (100 msec) after stimulus presentation. Surprisingly, disruption of the lateral occipital area resulted in increased performance scores and enhanced neural correlates of surface segregation. This facilitatory effect was also mainly found in an early time window (100 msec) after stimulus presentation. These results suggest a "push-pull" interaction in which dorsal and ventral extrastriate areas are being recruited or inhibited depending on stimulus category and task demands.

Seeing without knowing: neural signatures of perceptual inference in the absence of report.

Every day, we experience a rich and complex visual world. Our brain constantly translates meaningless fragmented input into coherent objects and scenes. However, our attentional capabilities are limited, and we can only report the few items that we happen to attend to. So what happens to items that are not cognitively accessed? Do these remain fragmentary and meaningless? Or are they processed up to a level where perceptual inferences take place about image composition? To investigate this, we recorded brain activity using fMRI while participants viewed images containing a Kanizsa figure, an illusion in which an object is perceived by means of perceptual inference. Participants were presented with the Kanizsa figure and three matched nonillusory control figures while they were engaged in an attentionally demanding distractor task. After the task, one group of participants was unable to identify the Kanizsa figure in a forced-choice decision task; hence, they were "inattentionally blind." A second group had no trouble identifying the Kanizsa figure. Interestingly, the neural signature that was unique to the processing of the Kanizsa figure was present in both groups. Moreover, within-subject multivoxel pattern analysis showed that the neural signature of unreported Kanizsa figures could be used to classify reported Kanizsa figures and that this cross-report classification worked better for the Kanizsa condition than for the control conditions. Together, these results suggest that stimuli that are not cognitively accessed are processed up to levels of perceptual interpretation.

Accurate metacognition for visual sensory memory representations.

The capacity to attend to multiple objects in the visual field is limited. However, introspectively, people feel that they see the whole visual world at once. Some scholars suggest that this introspective feeling is based on short-lived sensory memory representations, whereas others argue that the feeling of seeing more than can be attended to is illusory. Here, we investigated this phenomenon by combining objective memory performance with subjective confidence ratings during a change-detection task. This allowed us to compute a measure of metacognition--the degree of knowledge that subjects have about the correctness of their decisions--for different stages of memory. We show that subjects store more objects in sensory memory than they can attend to but, at the same time, have similar metacognition for sensory memory and working memory representations. This suggests that these subjective impressions are not an illusion but accurate reflections of the richness of visual perception.

How awareness changes the relative weights of evidence during human decision-making.

Human decisions are based on accumulating evidence over time for different options. Here we ask a simple question: How is the accumulation of evidence affected by the level of awareness of the information? We examined the influence of awareness on decision-making using combined behavioral methods and magneto-encephalography (MEG). Participants were required to make decisions by accumulating evidence over a series of visually presented arrow stimuli whose visibility was modulated by masking. Behavioral results showed that participants could accumulate evidence under both high and low visibility. However, a top-down strategic modulation of the flow of incoming evidence was only present for stimuli with high visibility: once enough evidence had been accrued, participants strategically reduced the impact of new incoming stimuli. Also, decision-making speed and confidence were strongly modulated by the strength of the evidence for high-visible but not low-visible evidence, even though direct priming effects were identical for both types of stimuli. Neural recordings revealed that, while initial perceptual processing was independent of visibility, there was stronger top-down amplification for stimuli with high visibility than low visibility. Furthermore, neural markers of evidence accumulation over occipito-parietal cortex showed a strategic bias only for highly visible sensory information, speeding up processing and reducing neural computations related to the decision process. Our results indicate that the level of awareness of information changes decision-making: while accumulation of evidence already exists under low visibility conditions, high visibility allows evidence to be accumulated up to a higher level, leading to important strategical top-down changes in decision-making. Our results therefore suggest a potential role of awareness in deploying flexible strategies for biasing information acquisition in line with one's expectations and goals.

Parallel development of ERP and behavioural measurements of visual segmentation.

Visual segmentation, a process in which elements are integrated into a form and segregated from the background, is known to differ from adults at infancy. The further developmental trajectory of this process, and of the underlying brain mechanisms, during childhood and adolescence is unknown. The aim of the study was to investigate the developmental trajectory of ERP reflections of visual segmentation, and to relate this to behavioural performance. One hundred and eleven typically developing children from 7 to 18 years of age were divided into six age groups. Each child performed two visual tasks. In a texture segmentation task, the difference in event-related potential (ERP) response to homogeneous (no visual segmentation) and checkered stimuli (visual segmentation) was investigated. In addition, behavioural performance on integration of elements into contours was measured. Both behavioural and ERP measurements of visual segmentation differed from adults in 7-12 year-old children. Behaviourally, young children were less able to integrate elements into a contour than older children. In addition, a developmental change was present in the ERP pattern evoked by homogeneous versus checkered stimuli. The largest differences in behaviour and ERPs were found between 7-8- and 9-10-, and between 11-12- and 13-14-year-old children, indicating the strongest development between those age groups. Behavioural as well as ERP measurements at 13-14 years of age showed similar results to those of adults. These results reveal that visual segmentation continues to develop until early puberty. Only by 13-14 years of age, children do integrate and segregate visual information as adults do. These results can be interpreted in terms of functional connectivity within the visual cortex.

Does perceptual learning require consciousness or attention?

It has been proposed that visual attention and consciousness are separate [Koch, C., & Tsuchiya, N. Attention and consciousness: Two distinct brain processes. Trends in Cognitive Sciences, 11, 16-22, 2007] and possibly even orthogonal processes [Lamme, V. A. F. Why visual attention and awareness are different. Trends in Cognitive Sciences, 7, 12-18, 2003]. Attention and consciousness converge when conscious visual percepts are attended and hence become available for conscious report. In such a view, a lack of reportability can have two causes: the absence of attention or the absence of a conscious percept. This raises an important question in the field of perceptual learning. It is known that learning can occur in the absence of reportability [Gutnisky, D. A., Hansen, B. J., Iliescu, B. F., & Dragoi, V. Attention alters visual plasticity during exposure-based learning. Current Biology, 19, 555-560, 2009; Seitz, A. R., Kim, D., & Watanabe, T. Rewards evoke learning of unconsciously processed visual stimuli in adult humans. Neuron, 61, 700-707, 2009; Seitz, A. R., & Watanabe, T. Is subliminal learning really passive? Nature, 422, 36, 2003; Watanabe, T., Náñez, J. E., & Sasaki, Y. Perceptual learning without perception. Nature, 413, 844-848, 2001], but it is unclear which of the two ingredients-consciousness or attention-is not necessary for learning. We presented textured figure-ground stimuli and manipulated reportability either by masking (which only interferes with consciousness) or with an inattention paradigm (which only interferes with attention). During the second session (24 hr later), learning was assessed neurally and behaviorally, via differences in figure-ground ERPs and via a detection task. Behavioral and neural learning effects were found for stimuli presented in the inattention paradigm and not for masked stimuli. Interestingly, the behavioral learning effect only became apparent when performance feedback was given on the task to measure learning, suggesting that the memory trace that is formed during inattention is latent until accessed. The results suggest that learning requires consciousness, and not attention, and further strengthen the idea that consciousness is separate from attention.

Confuse your illusion: feedback to early visual cortex contributes to perceptual completion.

A striking example of the constructive nature of visual perception is how the human visual system completes contours of occluded objects. To date, it is unclear whether perceptual completion emerges during early stages of visual processing or whether higher-level mechanisms are necessary. To answer this question, we used transcranial magnetic stimulation to disrupt signaling in V1/V2 and in the lateral occipital (LO) area at different moments in time while participants performed a discrimination task involving a Kanizsa-type illusory figure. Results show that both V1/V2 and higher-level visual area LO are critically involved in perceptual completion. However, these areas seem to be involved in an inverse hierarchical fashion, in which the critical time window for V1/V2 follows that for LO. These results are in line with the growing evidence that feedback to V1/V2 contributes to perceptual completion.

Spatially pooled contrast responses predict neural and perceptual similarity of naturalistic image categories.

The visual world is complex and continuously changing. Yet, our brain transforms patterns of light falling on our retina into a coherent percept within a few hundred milliseconds. Possibly, low-level neural responses already carry substantial information to facilitate rapid characterization of the visual input. Here, we computationally estimated low-level contrast responses to computer-generated naturalistic images, and tested whether spatial pooling of these responses could predict image similarity at the neural and behavioral level. Using EEG, we show that statistics derived from pooled responses explain a large amount of variance between single-image evoked potentials (ERPs) in individual subjects. Dissimilarity analysis on multi-electrode ERPs demonstrated that large differences between images in pooled response statistics are predictive of more dissimilar patterns of evoked activity, whereas images with little difference in statistics give rise to highly similar evoked activity patterns. In a separate behavioral experiment, images with large differences in statistics were judged as different categories, whereas images with little differences were confused. These findings suggest that statistics derived from low-level contrast responses can be extracted in early visual processing and can be relevant for rapid judgment of visual similarity. We compared our results with two other, well- known contrast statistics: Fourier power spectra and higher-order properties of contrast distributions (skewness and kurtosis). Interestingly, whereas these statistics allow for accurate image categorization, they do not predict ERP response patterns or behavioral categorization confusions. These converging computational, neural and behavioral results suggest that statistics of pooled contrast responses contain information that corresponds with perceived visual similarity in a rapid, low-level categorization task.