When mind and body align: examining the role of cross-modal congruency in conscious representations of happy facial expressions.

This study explored how congruency between facial mimicry and observed expressions affects the stability of conscious facial expression representations. Focusing on the congruency effect between proprioceptive/sensorimotor signals and visual stimuli for happy expressions, participants underwent a binocular rivalry task displaying neutral and happy faces. Mimicry was either facilitated with a chopstick or left unrestricted. Key metrics included Initial Percept (bias indicator), Onset Resolution Time (time from onset to Initial Percept), and Cumulative Time (content stabilization measure). Results indicated that mimicry manipulation significantly impacted Cumulative Time for happy faces, highlighting the importance of congruent mimicry in stabilizing conscious awareness of facial expressions. This supports embodied cognition models, showing the integration of proprioceptive information significantly biases conscious visual perception of facial expressions.

Are Color Experiences the Same across the Visual Field?

It seems obvious to laypeople that neurotypical humans experience color equivalently across their entire visual field. To some neuroscientists, psychologists, and philosophers, though, this claim has been met with skepticism, as neurophysiological evidence indicates the mechanisms that support color perception degrade with eccentricity. However, the argument that this entails altered color experience in peripheral vision is not universally accepted. Here, we address whether color experience is essentially equivalent between central and peripheral vision. To assess this, we will obtain similarity relationships between color experiences across the visual field using both online and laboratory-based far-field displays, while removing the confounds of saccades, memory, and expectation about color experiences. Our experiment was designed to provide clear evidence that would favor either unchanged or altered color experience relationships in the periphery. Our results are consistent with lay people's phenomenological reports: Color experiences, as probed by similarity relationships in central vision and the far field (60°), are equivalent when elicited by large stimuli. These findings challenge the widespread view in philosophy and cognitive science that peripheral color experiences are illusory, and are discussed in the context of their related neurophysiological, psychophysical, and philosophical literature.

Neural and computational processes of accelerated perceptual awareness and decisions: A 7T fMRI study.

Rapidly detecting salient information in our environments is critical for survival. Visual processing in subcortical areas like the pulvinar and amygdala has been shown to facilitate unconscious processing of salient stimuli. It is unknown, however, if and how these areas might interact with cortical regions to facilitate faster conscious perception of salient stimuli. Here we investigated these neural processes using 7T functional magnetic resonance imaging (fMRI) in concert with computational modelling while participants (n = 33) engaged in a breaking continuous flash suppression paradigm (bCFS) in which fearful and neutral faces are initially suppressed from conscious perception but then eventually 'breakthrough' into awareness. Participants reported faster breakthrough times for fearful faces compared with neutral faces. Drift-diffusion modelling suggested that perceptual evidence was accumulated at a faster rate for fearful faces compared with neutral faces. For both neutral and fearful faces, faster response times were associated with greater activity in the amygdala (specifically within its subregions, including superficial, basolateral and amygdalo-striatal transition area) and the insula. Faster rates of evidence accumulation coincided with greater activity in frontoparietal regions and occipital lobe, as well as the amygdala. A lower decision-boundary correlated with activity in the insula and the posterior cingulate cortex (PCC), but not with the amygdala. Overall, our findings suggest that hastened perceptual awareness of salient stimuli recruits the amygdala and, more specifically, is driven by accelerated evidence accumulation in fronto-parietal and visual areas. In sum, we have mapped distinct neural computations that accelerate perceptual awareness of visually suppressed faces.

Expectation and attention increase the integration of top-down and bottom-up signals in perception through different pathways.

Perception likely results from the interplay between sensory information and top-down signals. In this electroencephalography (EEG) study, we utilised the hierarchical frequency tagging (HFT) method to examine how such integration is modulated by expectation and attention. Using intermodulation (IM) components as a measure of nonlinear signal integration, we show in three different experiments that both expectation and attention enhance integration between top-down and bottom-up signals. Based on a multispectral phase coherence (MSPC) measure, we present two direct physiological measures to demonstrate the distinct yet related mechanisms of expectation and attention, which would not have been possible using other amplitude-based measures. Our results link expectation to the modulation of descending signals and to the integration of top-down and bottom-up information at lower levels of the visual hierarchy. Meanwhile, the results link attention to the modulation of ascending signals and to the integration of information at higher levels of the visual hierarchy. These results are consistent with the predictive coding account of perception.

Integrated information structure collapses with anesthetic loss of conscious arousal in Drosophila melanogaster.

The physical basis of consciousness remains one of the most elusive concepts in current science. One influential conjecture is that consciousness is to do with some form of causality, measurable through information. The integrated information theory of consciousness (IIT) proposes that conscious experience, filled with rich and specific content, corresponds directly to a hierarchically organised, irreducible pattern of causal interactions; i.e. an integrated informational structure among elements of a system. Here, we tested this conjecture in a simple biological system (fruit flies), estimating the information structure of the system during wakefulness and general anesthesia. Consistent with this conjecture, we found that integrated interactions among populations of neurons during wakefulness collapsed to isolated clusters of interactions during anesthesia. We used classification analysis to quantify the accuracy of discrimination between wakeful and anesthetised states, and found that informational structures inferred conscious states with greater accuracy than a scalar summary of the structure, a measure which is generally championed as the main measure of IIT. In stark contrast to a view which assumes feedforward architecture for insect brains, especially fly visual systems, we found rich information structures, which cannot arise from purely feedforward systems, occurred across the fly brain. Further, these information structures collapsed uniformly across the brain during anesthesia. Our results speak to the potential utility of the novel concept of an "informational structure" as a measure for level of consciousness, above and beyond simple scalar values.

Enriched category as a model of qualia structure based on similarity judgements.

Qualitative relationships between two instances of conscious experiences can be quantified through the perceived similarity. Previously, we proposed that by defining similarity relationships as arrows and conscious experiences as objects, we can define a category of qualia in the context of category theory. However, the example qualia categories we proposed were highly idealized and limited to cases where perceived similarity is binary: either present or absent without any gradation. Here, we introduce enriched category theory to address the graded levels of similarity that arises in many instances of qualia. Enriched categories generalize the concept of a relation between objects as a directed arrow (or morphism) in ordinary category theory to a more flexible notion, such as a measure of distance. As an alternative relation, here we propose a graded measure of perceived dissimilarity between the two objects. We claim that enriched categories accommodate various types of conscious experiences. An important consequence of this claim is the application of the Yoneda lemma in enriched category; we can characterize a quale through a collection of relationships between the quale and the other qualia up to an (enriched) isomorphism.

Emergence of Integrated Information at Macro Timescales in Real Neural Recordings.

How a system generates conscious experience remains an elusive question. One approach towards answering this is to consider the information available in the system from the perspective of the system itself. Integrated information theory (IIT) proposes a measure to capture this integrated information (Φ). While Φ can be computed at any spatiotemporal scale, IIT posits that it be applied at the scale at which the measure is maximised. Importantly, Φ in conscious systems should emerge to be maximal not at the smallest spatiotemporal scale, but at some macro scale where system elements or timesteps are grouped into larger elements or timesteps. Emergence in this sense has been demonstrated in simple example systems composed of logic gates, but it remains unclear whether it occurs in real neural recordings which are generally continuous and noisy. Here we first utilise a computational model to confirm that Φ becomes maximal at the temporal scales underlying its generative mechanisms. Second, we search for emergence in local field potentials from the fly brain recorded during wakefulness and anaesthesia, finding that normalised Φ (wake/anaesthesia), but not raw Φ values, peaks at 5 ms. Lastly, we extend our model to investigate why raw Φ values themselves did not peak. This work extends the application of Φ to simple artificial systems consisting of logic gates towards searching for emergence of a macro spatiotemporal scale in real neural systems.

Conscious perception of flickering stimuli in binocular rivalry and continuous flash suppression is not affected by tACS-induced SSR modulation.

The content of conscious perception is known to correlate with steady-state responses (SSRs), yet their causal relationship remains unclear. Can we manipulate conscious perception by directly interfering with SSRs through transcranial alternating current stimulation (tACS)? Here, we directly addressed this question in three experiments involving binocular rivalry and continuous flash suppression (CFS). Specifically, while participants (N = 24) viewed either binocular rivalry or tried to detect stimuli masked by CFS, we applied sham or real tACS across parieto-occipital cortex at either the same or a different frequency and phase as an SSR eliciting flicker stimulus. We found that tACS did not differentially affect conscious perception in the forms of predominance, CFS detection accuracy, reaction time, or metacognitive sensitivity, confirmed by Bayesian statistics. We conclude that tACS application at frequencies of stimulus-induced SSRs does not have perceptual effects and that SSRs may be epiphenomenal to conscious perception.

From intermodulation components to visual perception and cognition-a review.

Perception results from complex interactions among sensory and cognitive processes across hierarchical levels in the brain. Intermodulation (IM) components, used in frequency tagging neuroimaging designs, have emerged as a promising direct measure of such neural interactions. IMs have initially been used in electroencephalography (EEG) to investigate low-level visual processing. In a more recent trend, IMs in EEG and other neuroimaging methods are being used to shed light on mechanisms of mid- and high-level perceptual processes, including the involvement of cognitive functions such as attention and expectation. Here, we provide an account of various mechanisms that may give rise to IMs in neuroimaging data, and what these IMs may look like. We discuss methodologies that can be implemented for different uses of IMs and we demonstrate how IMs can provide insights into the existence, the degree and the type of neural integration mechanisms at hand. We then review a range of recent studies exploiting IMs in visual perception research, placing an emphasis on high-level vision and the influence of awareness and cognition on visual processing. We conclude by suggesting future directions that can enhance the benefits of IM-methodology in perception research.

Unified framework for information integration based on information geometry.

Assessment of causal influences is a ubiquitous and important subject across diverse research fields. Drawn from consciousness studies, integrated information is a measure that defines integration as the degree of causal influences among elements. Whereas pairwise causal influences between elements can be quantified with existing methods, quantifying multiple influences among many elements poses two major mathematical difficulties. First, overestimation occurs due to interdependence among influences if each influence is separately quantified in a part-based manner and then simply summed over. Second, it is difficult to isolate causal influences while avoiding noncausal confounding influences. To resolve these difficulties, we propose a theoretical framework based on information geometry for the quantification of multiple causal influences with a holistic approach. We derive a measure of integrated information, which is geometrically interpreted as the divergence between the actual probability distribution of a system and an approximated probability distribution where causal influences among elements are statistically disconnected. This framework provides intuitive geometric interpretations harmonizing various information theoretic measures in a unified manner, including mutual information, transfer entropy, stochastic interaction, and integrated information, each of which is characterized by how causal influences are disconnected. In addition to the mathematical assessment of consciousness, our framework should help to analyze causal relationships in complex systems in a complete and hierarchical manner.

Are the Neural Correlates of Consciousness in the Front or in the Back of the Cerebral Cortex? Clinical and Neuroimaging Evidence.

The role of the frontal cortex in consciousness remains a matter of debate. In this Perspective, we will critically review the clinical and neuroimaging evidence for the involvement of the front versus the back of the cortex in specifying conscious contents and discuss promising research avenues.Dual Perspectives Companion Paper: Should a Few Null Findings Falsify Prefrontal Theories of Conscious Perception?, by Brian Odegaard, Robert T. Knight, and Hakwan Lau.

Intracranial markers of conscious face perception in humans.

Investigations of the neural basis of consciousness have greatly benefited from protocols that involve the presentation of stimuli at perceptual threshold, enabling the assessment of the patterns of brain activity that correlate with conscious perception, independently of any changes in sensory input. However, the comparison between perceived and unperceived trials would be expected to reveal not only the core neural substrate of a particular conscious perception, but also aspects of brain activity that facilitate, hinder or tend to follow conscious perception. We take a step towards the resolution of these confounds by combining an analysis of neural responses observed during the presentation of faces partially masked by Continuous Flash Suppression, and those responses observed during the unmasked presentation of faces and other images in the same subjects. We employed multidimensional classifiers to decode physical properties of stimuli or perceptual states from spectrotemporal representations of electrocorticographic signals (1071 channels in 5 subjects). Neural activity in certain face responsive areas located in both the fusiform gyrus and in the lateral-temporal/inferior-parietal cortex discriminated seen vs. unseen faces in the masked paradigm and upright faces vs. other categories in the unmasked paradigm. However, only the former discriminated upright vs. inverted faces in the unmasked paradigm. Our results suggest a prominent role for the fusiform gyrus in the configural perception of faces, and possibly other objects that are holistically processed. More generally, we advocate comparative analysis of neural recordings obtained during different, but related, experimental protocols as a promising direction towards elucidating the functional specificities of the patterns of neural activation that accompany our conscious experiences.

Measuring Integrated Information from the Decoding Perspective.

Accumulating evidence indicates that the capacity to integrate information in the brain is a prerequisite for consciousness. Integrated Information Theory (IIT) of consciousness provides a mathematical approach to quantifying the information integrated in a system, called integrated information, Φ. Integrated information is defined theoretically as the amount of information a system generates as a whole, above and beyond the amount of information its parts independently generate. IIT predicts that the amount of integrated information in the brain should reflect levels of consciousness. Empirical evaluation of this theory requires computing integrated information from neural data acquired from experiments, although difficulties with using the original measure Φ precludes such computations. Although some practical measures have been previously proposed, we found that these measures fail to satisfy the theoretical requirements as a measure of integrated information. Measures of integrated information should satisfy the lower and upper bounds as follows: The lower bound of integrated information should be 0 and is equal to 0 when the system does not generate information (no information) or when the system comprises independent parts (no integration). The upper bound of integrated information is the amount of information generated by the whole system. Here we derive the novel practical measure Φ* by introducing a concept of mismatched decoding developed from information theory. We show that Φ* is properly bounded from below and above, as required, as a measure of integrated information. We derive the analytical expression of Φ* under the Gaussian assumption, which makes it readily applicable to experimental data. Our novel measure Φ* can generally be used as a measure of integrated information in research on consciousness, and also as a tool for network analysis on diverse areas of biology.

Frequency-dependent spatiotemporal profiles of visual responses recorded with subdural ECoG electrodes in awake monkeys: Differences between high- and low-frequency activity.

Electrocorticography (ECoG) constitutes a powerful and promising neural recording modality in humans and animals. ECoG signals are often decomposed into several frequency bands, among which the so-called high-gamma band (80-250Hz) has been proposed to reflect local cortical functions near the cortical surface below the ECoG electrodes. It is typically assumed that the lower the frequency bands, the lower the spatial resolution of the signals; thus, there is not much to gain by analyzing the event-related changes of the ECoG signals in the lower-frequency bands. However, differences across frequency bands have not been systematically investigated. To address this issue, we recorded ECoG activity from two awake monkeys performing a retinotopic mapping task. We characterized the spatiotemporal profiles of the visual responses in the time-frequency domain. We defined the preferred spatial position, receptive field (RF), and response latencies of band-limited power (BLP) (i.e., alpha [3.9-11.7Hz], beta [15.6-23.4Hz], low [30-80Hz] and high [80-250Hz] gamma) for each electrode and compared them across bands and time-domain visual evoked potentials (VEPs). At the population level, we found that the spatial preferences were comparable across bands and VEPs. The high-gamma power showed a smaller RF than the other bands and VEPs. The response latencies for the alpha band were always longer than the latencies for the other bands and fastest in VEPs. Comparing the response profiles in both space and time for each cortical region (V1, V4+, and TEO/TE) revealed regional idiosyncrasies. Although the latencies of visual responses in the beta, low-, and high-gamma bands were almost identical in V1 and V4+, beta and low-gamma BLP occurred about 17ms earlier than high-gamma power in TEO/TE. Furthermore, TEO/TE exhibited a unique pattern in the spatial response profile: the alpha and high-gamma responses tended to prefer the foveal regions, whereas the beta and low-gamma responses preferred the peripheral visual fields with larger RFs. This suggests that neurons in TEO/TE first receive less selective spatial information via beta and low-gamma BLP but later receive more fine-tuned spatial foveal information via high-gamma power. This result is consistent with a hypothesis previously proposed by Nakamura et al. (1993) that states that visual processing in TEO/TE starts with coarse-grained information, which primes subsequent fine-grained information. Collectively, our results demonstrate that ECoG can be a potent tool for investigating the nature of the neural computations in each cortical region that cannot be fully understood by measuring only the spiking activity, through the incorporation of the knowledge of the spatiotemporal characteristics across all frequency bands.

Large Capacity of Conscious Access for Incidental Memories in Natural Scenes.

When searching a crowd, people can detect a target face only by direct fixation and attention. Once the target is found, it is consciously experienced and remembered, but what is the perceptual fate of the fixated nontarget faces? Whereas introspection suggests that one may remember nontargets, previous studies have proposed that almost no memory should be retained. Using a gaze-contingent paradigm, we asked subjects to visually search for a target face within a crowded natural scene and then tested their memory for nontarget faces, as well as their confidence in those memories. Subjects remembered up to seven fixated, nontarget faces with more than 70% accuracy. Memory accuracy was correlated with trial-by-trial confidence ratings, which implies that the memory was consciously maintained and accessed. When the search scene was inverted, no more than three nontarget faces were remembered. These findings imply that incidental memory for faces, such as those recalled by eyewitnesses, is more reliable than is usually assumed.

Feedforward connectivity patterns from visual areas to the front of the brain contain information about sensory stimuli regardless of awareness or report.

Current theories of consciousness can be categorized to some extent by their predictions about the putative role of the prefrontal cortex (PFC) in conscious perception. One family of the theories proposes that the PFC is necessary for conscious perception. The other postulates that the PFC is not necessary and that other areas (e.g., posterior cortical areas) are more important for conscious perception. No-report paradigms could potentially arbitrate the debate as they disentangle task reporting from conscious perception. While previous no-report paradigms tend to point to a reduction in PFC activity, they have not examined the critical role of the PFC in "monitoring" or "reading out" the patterns of activity in the sensory cortex to generate conscious perception. To address this, we reanalysed electroencephalography (EEG) data from a no-report inattentional blindness paradigm (Shafto & Pitts, 2015). We examined the role of feedforward input patterns to the PFC from sensory cortices. We employed nonparametric spectral Granger causality and quantified the amount of information that reflected the contents of consciousness using multivariate classifiers. Unexpectedly, regardless of whether the stimulus was consciously seen or not, we found that information relating to the current sensory stimulus was present in the pattern of inputs from visual areas to the PFC. In light of these findings, we suggest various theories of consciousness need to be revised to accommodate the fact that the contents of consciousness are decodable from the input patterns from posterior sensory regions to the PFC, regardless of awareness (or report).

Conscious perception of fear in faces: Insights from high-density EEG and perceptual awareness scale with threshold stimuli.

Contrary to the extensive research on processing subliminal and/or unattended emotional facial expressions, only a minority of studies have investigated the neural correlates of consciousness (NCCs) of emotions conveyed by faces. In the present high-density electroencephalography (EEG) study, we first employed a staircase procedure to identify each participant's perceptual threshold of the emotion expressed by the face and then compared the EEG signals elicited in trials where the participants were aware with the activity elicited in trials where participants were unaware of the emotions expressed by these, otherwise identical, faces. Drawing on existing knowledge of the neural mechanisms of face processing and NCCs, we hypothesized that activity in frontal electrodes would be modulated in relation to participants' awareness of facial emotional content. More specifically, we hypothesized that the NCC of fear seen on someone else's face could be detected as a modulation of a later and more anterior (i.e., at frontal sites) event-related potential (ERP) than the face-sensitive N170. By adopting a data-driven approach and cluster-based statistics to the analysis of EEG signals, the results were clear-cut in showing that visual awareness of fear was associated with the modulation of a frontal ERP component in a 150-300 msec interval. These insights are dissected and contextualized in relation to prevailing theories of visual consciousness and their proposed NCC benchmarks.

Opposing effects of attention and consciousness on afterimages.

The brain's ability to handle sensory information is influenced by both selective attention and consciousness. There is no consensus on the exact relationship between these two processes and whether they are distinct. So far, no experiment has simultaneously manipulated both. We carried out a full factorial 2 x 2 study of the simultaneous influences of attention and consciousness (as assayed by visibility) on perception, correcting for possible concurrent changes in attention and consciousness. We investigated the duration of afterimages for all four combinations of high versus low attention and visible versus invisible. We show that selective attention and visual consciousness have opposite effects: paying attention to the grating decreases the duration of its afterimage, whereas consciously seeing the grating increases the afterimage duration. These findings provide clear evidence for distinctive influences of selective attention and consciousness on visual perception.

Separating weak integrated information theory into inspired and aspirational approaches.

Mediano et al. (The strength of weak integrated information theory. Trends Cogn Sci 2022;26: 646-55.) separate out strong and weak flavours of the integrated information theory (IIT) of consciousness. They describe 'strong IIT' as attempting to derive a universal formula for consciousness and 'weak IIT' as searching for empirically measurable correlates of aspects of consciousness. We put forward that their overall notion of 'weak IIT' may be too weak. Rather, it should be separated out to distinguish 'aspirational-IIT', which aims to empirically test IIT by making trade-offs to its proposed measures, and 'IIT-inspired' approaches, which adopt high-level ideas of IIT while dropping the mathematical framework it reaches through its introspective, first-principles approach to consciousness.