Conscious perception of fear in faces: Insights from high-density EEG and perceptual awareness scale with threshold stimuli.

Contrary to the extensive research on processing subliminal and/or unattended emotional facial expressions, only a minority of studies have investigated the neural correlates of consciousness (NCCs) of emotions conveyed by faces. In the present high-density electroencephalography (EEG) study, we first employed a staircase procedure to identify each participant's perceptual threshold of the emotion expressed by the face and then compared the EEG signals elicited in trials where the participants were aware with the activity elicited in trials where participants were unaware of the emotions expressed by these, otherwise identical, faces. Drawing on existing knowledge of the neural mechanisms of face processing and NCCs, we hypothesized that activity in frontal electrodes would be modulated in relation to participants' awareness of facial emotional content. More specifically, we hypothesized that the NCC of fear seen on someone else's face could be detected as a modulation of a later and more anterior (i.e., at frontal sites) event-related potential (ERP) than the face-sensitive N170. By adopting a data-driven approach and cluster-based statistics to the analysis of EEG signals, the results were clear-cut in showing that visual awareness of fear was associated with the modulation of a frontal ERP component in a 150-300 msec interval. These insights are dissected and contextualized in relation to prevailing theories of visual consciousness and their proposed NCC benchmarks.

When mind and body align: examining the role of cross-modal congruency in conscious representations of happy facial expressions.

This study explored how congruency between facial mimicry and observed expressions affects the stability of conscious facial expression representations. Focusing on the congruency effect between proprioceptive/sensorimotor signals and visual stimuli for happy expressions, participants underwent a binocular rivalry task displaying neutral and happy faces. Mimicry was either facilitated with a chopstick or left unrestricted. Key metrics included Initial Percept (bias indicator), Onset Resolution Time (time from onset to Initial Percept), and Cumulative Time (content stabilization measure). Results indicated that mimicry manipulation significantly impacted Cumulative Time for happy faces, highlighting the importance of congruent mimicry in stabilizing conscious awareness of facial expressions. This supports embodied cognition models, showing the integration of proprioceptive information significantly biases conscious visual perception of facial expressions.

Feedforward connectivity patterns from visual areas to the front of the brain contain information about sensory stimuli regardless of awareness or report.

Current theories of consciousness can be categorized to some extent by their predictions about the putative role of the prefrontal cortex (PFC) in conscious perception. One family of the theories proposes that the PFC is necessary for conscious perception. The other postulates that the PFC is not necessary and that other areas (e.g., posterior cortical areas) are more important for conscious perception. No-report paradigms could potentially arbitrate the debate as they disentangle task reporting from conscious perception. While previous no-report paradigms tend to point to a reduction in PFC activity, they have not examined the critical role of the PFC in "monitoring" or "reading out" the patterns of activity in the sensory cortex to generate conscious perception. To address this, we reanalysed electroencephalography (EEG) data from a no-report inattentional blindness paradigm (Shafto & Pitts, 2015). We examined the role of feedforward input patterns to the PFC from sensory cortices. We employed nonparametric spectral Granger causality and quantified the amount of information that reflected the contents of consciousness using multivariate classifiers. Unexpectedly, regardless of whether the stimulus was consciously seen or not, we found that information relating to the current sensory stimulus was present in the pattern of inputs from visual areas to the PFC. In light of these findings, we suggest various theories of consciousness need to be revised to accommodate the fact that the contents of consciousness are decodable from the input patterns from posterior sensory regions to the PFC, regardless of awareness (or report).

When philosophical nuance matters: safeguarding consciousness research from restrictive assumptions.

In this paper, we revisit the debate surrounding the Unfolding Argument (UA) against causal structure theories of consciousness (as well as the hard-criteria research program it prescribes), using it as a platform for discussing theoretical and methodological issues in consciousness research. Causal structure theories assert that consciousness depends on a particular causal structure of the brain. Our claim is that some of the assumptions fueling the UA are not warranted, and therefore we should reject the methodology for consciousness science that the UA prescribes. First, we briefly survey the most popular philosophical positions in consciousness science, namely physicalism and functionalism. We discuss the relations between these positions and the behaviorist methodology that the UA assumptions express, despite the contrary claim of its proponents. Second, we argue that the same reasoning that the UA applies against causal structure theories can be applied to functionalist approaches, thus proving too much and deeming as unscientific a whole range of (non-causal structure) theories. Since this is overly restrictive and fits poorly with common practice in cognitive neuroscience, we suggest that the reasoning of the UA must be flawed. Third, we assess its philosophical assumptions, which express a restrictive methodology, and conclude that there are reasons to reject them. Finally, we propose a more inclusive methodology for consciousness science, that includes neural, behavioral, and phenomenological evidence (provided by the first-person perspective) without which consciousness science could not even start. Then, we extend this discussion to the scope of consciousness science, and conclude that theories of consciousness should be tested and evaluated on humans, and not on systems considerably different from us. Rather than restricting the methodology of consciousness science, we should, at this point, restrict the range of systems upon which it is supposed to be built.

INs and OUTs of faces in consciousness: a study of the temporal evolution of consciousness of faces during binocular rivalry.

Contents of consciousness change over time. However, the study of dynamics in consciousness has been largely neglected. Aru and Bachmann have recently brought to the attention of scientists dealing with consciousness the relevance of making inquiries about its temporal evolution. Importantly, they also pointed out several experimental questions as guidelines for researchers interested in studying the temporal evolution of consciousness, including the phases of formation and dissolution of content. They also suggested that these two phases could be characterized by asymmetric inertia. The main objective of the present investigation was to approximate the dynamics of these two phases in the context of conscious face perception. To this aim, we tested the time course of content transitions during a binocular rivalry task using face stimuli and asked participants to map their subjective experience of transitions from one content to the other through a joystick. We then computed metrics of joystick velocity linked to content transitions as proxies of the formation and dissolution phases. We found a general phase effect such that the formation phase was slower than the dissolution phase. Furthermore, we observed an effect specific to happy facial expressions, such that their contents were slower to form and dissolve than that of neutral expressions. We further propose to include a third phase of stabilization of conscious content between formation and dissolution.

When do parts form wholes? Integrated information as the restriction on mereological composition.

Under what conditions are material objects, such as particles, parts of a whole object? This is the composition question and is a longstanding open question in philosophy. Existing attempts to specify a non-trivial restriction on composition tend to be vague and face serious counterexamples. Consequently, two extreme answers have become mainstream: composition (the forming of a whole by its parts) happens under no or all conditions. In this paper, we provide a self-contained introduction to the integrated information theory (IIT) of consciousness. We show that IIT specifies a non-trivial restriction on composition: composition happens when integrated information is maximized. We compare the IIT restriction to existing proposals and argue that the IIT restriction has significant advantages, especially in response to the problems of vagueness and counterexamples. An appendix provides an introduction to calculating parts and wholes with a simple system.

Separating weak integrated information theory into inspired and aspirational approaches.

Mediano et al. (The strength of weak integrated information theory. Trends Cogn Sci 2022;26: 646-55.) separate out strong and weak flavours of the integrated information theory (IIT) of consciousness. They describe 'strong IIT' as attempting to derive a universal formula for consciousness and 'weak IIT' as searching for empirically measurable correlates of aspects of consciousness. We put forward that their overall notion of 'weak IIT' may be too weak. Rather, it should be separated out to distinguish 'aspirational-IIT', which aims to empirically test IIT by making trade-offs to its proposed measures, and 'IIT-inspired' approaches, which adopt high-level ideas of IIT while dropping the mathematical framework it reaches through its introspective, first-principles approach to consciousness.

A Quantum Geometric Framework for Modeling Color Similarity Judgments.

Since Tversky argued that similarity judgments violate the three metric axioms, asymmetrical similarity judgments have been particularly challenging for standard, geometric models of similarity, such as multidimensional scaling. According to Tversky, asymmetrical similarity judgments are driven by differences in salience or extent of knowledge. However, the notion of salience has been difficult to operationalize, especially for perceptual stimuli for which there are no apparent differences in extent of knowledge. To investigate similarity judgments between perceptual stimuli, across three experiments, we collected data where individuals would rate the similarity of a pair of temporally separated color patches. We identified several violations of symmetry in the empirical results, which the conventional multidimensional scaling model cannot readily capture. Pothos et al. proposed a quantum geometric model of similarity to account for Tversky's findings. In the present work, we extended this model to a more general framework that can be fit to similarity judgments. We fitted several variants of quantum and multidimensional scaling models to the behavioral data and concluded in favor of the quantum approach. Without further modifications of the model, the best-fit quantum model additionally predicted violations of the triangle inequality that we observed in the same data. Overall, by offering a different form of geometric representation, the quantum geometric framework of similarity provides a viable alternative to multidimensional scaling for modeling similarity judgments, while still allowing a convenient, spatial illustration of similarity.

Are Color Experiences the Same across the Visual Field?

It seems obvious to laypeople that neurotypical humans experience color equivalently across their entire visual field. To some neuroscientists, psychologists, and philosophers, though, this claim has been met with skepticism, as neurophysiological evidence indicates the mechanisms that support color perception degrade with eccentricity. However, the argument that this entails altered color experience in peripheral vision is not universally accepted. Here, we address whether color experience is essentially equivalent between central and peripheral vision. To assess this, we will obtain similarity relationships between color experiences across the visual field using both online and laboratory-based far-field displays, while removing the confounds of saccades, memory, and expectation about color experiences. Our experiment was designed to provide clear evidence that would favor either unchanged or altered color experience relationships in the periphery. Our results are consistent with lay people's phenomenological reports: Color experiences, as probed by similarity relationships in central vision and the far field (60°), are equivalent when elicited by large stimuli. These findings challenge the widespread view in philosophy and cognitive science that peripheral color experiences are illusory, and are discussed in the context of their related neurophysiological, psychophysical, and philosophical literature.

Unifying pairwise interactions in complex dynamics.

Scientists have developed hundreds of techniques to measure the interactions between pairs of processes in complex systems, but these computational methods-from contemporaneous correlation coefficients to causal inference methods-define and formulate interactions differently, using distinct quantitative theories that remain largely disconnected. Here we introduce a large assembled library of 237 statistics of pairwise interactions, and assess their behavior on 1,053 multivariate time series from a wide range of real-world and model-generated systems. Our analysis highlights commonalities between disparate mathematical formulations of interactions, providing a unified picture of a rich interdisciplinary literature. Using three real-world case studies, we then show that simultaneously leveraging diverse methods can uncover those most suitable for addressing a given problem, facilitating interpretable understanding of the quantitative formulation of pairwise dependencies that drive successful performance. Our results and accompanying software enable comprehensive analysis of time-series interactions by drawing on decades of diverse methodological contributions.

What can we experience and report on a rapidly presented image? Intersubjective measures of specificity of freely reported contents of consciousness.

Background: A majority of previous studies appear to support a view that human observers can only perceive coarse information from a natural scene image when it is presented rapidly (<100ms, masked). In these studies, participants were often forced to choose an answer from options that experimenters preselected. These options can underestimate what participants experience and can report on it. The current study aims to introduce a novel methodology to investigate how detailed information participants can report after briefly seeing a natural scene image.     Methods: We used a novel free-report paradigm to examine what people can freely report following a rapidly presented natural scene image (67/133/267ms, masked). N = 670 online participants typed up to five words to report what they saw in the image together with confidence of the respective responses. We developed a novel index, Intersubjective Agreement (IA). IA quantifies how specifically the response words were used to describe the target image, with a high value meaning the word is not often reported for other images. Importantly, IA eliminates the need for experimenters to preselect response options. Results: The words with high IA values are often something detailed (e.g., a small object) in a particular image. With IA, unlike commonly believed, we demonstrated that participants reported highly specific and detailed aspects of the briefly (even at 67ms, masked) shown image. Further, IA is positively correlated with confidence, indicating metacognitive conscious access to the reported aspects of the image. Conclusion: These new findings challenge the dominant view that the content of rapid scene experience is limited to global and coarse gist. Our novel paradigm opens a door to investigate various contents of consciousness with a free-report paradigm.

Beyond traditional sleep scoring: Massive feature extraction and data-driven clustering of sleep time series.

The widely used guidelines for sleep staging were developed for the visual inspection of electrophysiological recordings by the human eye. As such, these rules reflect a limited range of features in these data and are therefore restricted in accurately capturing the physiological changes associated with sleep. Here we present a novel analysis framework that extensively characterizes sleep dynamics using over 7700 time-series features from the hctsa software. We used clustering to categorize sleep epochs based on the similarity of their time-series features, without relying on established scoring conventions. The resulting sleep structure overlapped substantially with that defined by visual scoring. However, we also observed discrepancies between our approach and traditional scoring. This divergence principally stemmed from the extensive characterization by hctsa features, which captured distinctive time-series properties within the traditionally defined sleep stages that are overlooked with visual scoring. Lastly, we report time-series features that are highly discriminative of stages. Our framework lays the groundwork for a data-driven exploration of sleep sub-stages and has significant potential to identify new signatures of sleep disorders and conscious sleep states.

Surprising Threats Accelerate Conscious Perception.

The folk psychological notion that "we see what we expect to see" is supported by evidence that we become consciously aware of visual stimuli that match our prior expectations more quickly than stimuli that violate our expectations. Similarly, "we see what we want to see," such that more biologically-relevant stimuli are also prioritised for conscious perception. How, then, is perception shaped by biologically-relevant stimuli that we did not expect? Here, we conducted two experiments using breaking continuous flash suppression (bCFS) to investigate how prior expectations modulated response times to neutral and fearful faces. In both experiments, we found that prior expectations for neutral faces hastened responses, whereas the opposite was true for fearful faces. This interaction between emotional expression and prior expectations was driven predominantly by participants with higher trait anxiety. Electroencephalography (EEG) data collected in Experiment 2 revealed an interaction evident in the earliest stages of sensory encoding, suggesting prediction errors expedite sensory encoding of fearful faces. These findings support a survival hypothesis, where biologically-relevant fearful stimuli are prioritised for conscious access even more so when unexpected, especially for people with high trait anxiety.

Emergence of Integrated Information at Macro Timescales in Real Neural Recordings.

How a system generates conscious experience remains an elusive question. One approach towards answering this is to consider the information available in the system from the perspective of the system itself. Integrated information theory (IIT) proposes a measure to capture this integrated information (Φ). While Φ can be computed at any spatiotemporal scale, IIT posits that it be applied at the scale at which the measure is maximised. Importantly, Φ in conscious systems should emerge to be maximal not at the smallest spatiotemporal scale, but at some macro scale where system elements or timesteps are grouped into larger elements or timesteps. Emergence in this sense has been demonstrated in simple example systems composed of logic gates, but it remains unclear whether it occurs in real neural recordings which are generally continuous and noisy. Here we first utilise a computational model to confirm that Φ becomes maximal at the temporal scales underlying its generative mechanisms. Second, we search for emergence in local field potentials from the fly brain recorded during wakefulness and anaesthesia, finding that normalised Φ (wake/anaesthesia), but not raw Φ values, peaks at 5 ms. Lastly, we extend our model to investigate why raw Φ values themselves did not peak. This work extends the application of Φ to simple artificial systems consisting of logic gates towards searching for emergence of a macro spatiotemporal scale in real neural systems.

How much can we differentiate at a brief glance: revealing the truer limit in conscious contents through the massive report paradigm (MRP).

Upon a brief glance, how well can we differentiate what we see from what we do not? Previous studies answered this question as 'poorly'. This is in stark contrast with our everyday experience. Here, we consider the possibility that previous restriction in stimulus variability and response alternatives reduced what participants could express from what they consciously experienced. We introduce a novel massive report paradigm that probes the ability to differentiate what we see from what we do not. In each trial, participants viewed a natural scene image and judged whether a small image patch was a part of the original image. To examine the limit of discriminability, we also included subtler changes in the image as modification of objects. Neither the images nor patches were repeated per participant. Our results showed that participants were highly accurate (accuracy greater than 80%) in differentiating patches from the viewed images from patches that are not present. Additionally, the differentiation between original and modified objects was influenced by object sizes and/or the congruence between objects and the scene gists. Our massive report paradigm opens a door to quantitatively measure the limit of immense informativeness of a moment of consciousness.

Neural and computational processes of accelerated perceptual awareness and decisions: A 7T fMRI study.

Rapidly detecting salient information in our environments is critical for survival. Visual processing in subcortical areas like the pulvinar and amygdala has been shown to facilitate unconscious processing of salient stimuli. It is unknown, however, if and how these areas might interact with cortical regions to facilitate faster conscious perception of salient stimuli. Here we investigated these neural processes using 7T functional magnetic resonance imaging (fMRI) in concert with computational modelling while participants (n = 33) engaged in a breaking continuous flash suppression paradigm (bCFS) in which fearful and neutral faces are initially suppressed from conscious perception but then eventually 'breakthrough' into awareness. Participants reported faster breakthrough times for fearful faces compared with neutral faces. Drift-diffusion modelling suggested that perceptual evidence was accumulated at a faster rate for fearful faces compared with neutral faces. For both neutral and fearful faces, faster response times were associated with greater activity in the amygdala (specifically within its subregions, including superficial, basolateral and amygdalo-striatal transition area) and the insula. Faster rates of evidence accumulation coincided with greater activity in frontoparietal regions and occipital lobe, as well as the amygdala. A lower decision-boundary correlated with activity in the insula and the posterior cingulate cortex (PCC), but not with the amygdala. Overall, our findings suggest that hastened perceptual awareness of salient stimuli recruits the amygdala and, more specifically, is driven by accelerated evidence accumulation in fronto-parietal and visual areas. In sum, we have mapped distinct neural computations that accelerate perceptual awareness of visually suppressed faces.

Enriched category as a model of qualia structure based on similarity judgements.

Qualitative relationships between two instances of conscious experiences can be quantified through the perceived similarity. Previously, we proposed that by defining similarity relationships as arrows and conscious experiences as objects, we can define a category of qualia in the context of category theory. However, the example qualia categories we proposed were highly idealized and limited to cases where perceived similarity is binary: either present or absent without any gradation. Here, we introduce enriched category theory to address the graded levels of similarity that arises in many instances of qualia. Enriched categories generalize the concept of a relation between objects as a directed arrow (or morphism) in ordinary category theory to a more flexible notion, such as a measure of distance. As an alternative relation, here we propose a graded measure of perceived dissimilarity between the two objects. We claim that enriched categories accommodate various types of conscious experiences. An important consequence of this claim is the application of the Yoneda lemma in enriched category; we can characterize a quale through a collection of relationships between the quale and the other qualia up to an (enriched) isomorphism.

Unconscious Visual Working Memory: A critical review and Bayesian meta-analysis.

The relationship between consciousness and working memory (WM) has been recently debated both at the theoretical and methodological levels (Persuh et al., 2018; Velichkovsky, 2017). While there is behavioral and neural evidence that argues for the existence of unconscious WM, several methodological concerns have been raised, rendering this issue highly controversial. To address the robustness of the previous findings, here we adopt a meta-analytic approach to estimate the effect size and heterogeneity of the previously reported unconscious WM results, also including unpublished results. We used meta-regression to isolate relevant experimental variables, in particular, consciousness manipulation and the WM paradigm to identify the source of the heterogeneity in the reported effect size of the unconscious WM. Our meta-analysis supports the existence of the unconscious WM effect and critically reveals several experimental variables that contribute to relevant heterogeneity. Our analysis clarifies several theoretical and methodological issues. We recommend that future studies explicitly operationalize the definition of consciousness, standardize the methodology and systematically explore the role of critical variables for the unconscious WM effect.

An adjunction hypothesis between qualia and reports.

What are the nature of the relationship among qualia, contents of consciousness, and behavioral reports? Traditionally, this type of question has been only addressed via a qualitative and philosophical approach. Some theorists emphasize an incomplete and inaccurate nature of reports of one's own qualia to discourage formal research programs on qualia. Other empirical researchers, however, have made substantial progress in understanding the structure of qualia from such limited reports. What is the precise relationship between the two? To answer this question, we introduce the concept of "adjoint" or "adjunction" from the category theory in mathematics. We claim that the adjunction captures some aspects of the nuanced relationships between qualia and reports. The concept of adjunction allows us to clarify the conceptual issues with a precise mathematical formulation. In particular, adjunction establishes coherence between two categories that cannot be considered equivalent, yet has an important relationship. This rises in empirical experimental situations between qualia and reports. More importantly, an idea of adjunction naturally leads to various proposals of new empirical experiments to test the predictions about the nature of their relationship as well as other issues in consciousness research.