Implicit semantics gates visual awareness.

The current study asked whether impoverished peripheral vision led to perception immune from word-based semantic influences. We leveraged a peripheral sound-induced flash illusion. In each trial, two or three Mandarin characters were flashed quickly in the periphery with number-congruent or -incongruent beeps. We first successfully replicated the original illusions, showing auditory dominance. For example, when three characters were presented together with two beeps, the observer reported perceiving only two characters. Similarly, an additional beep induced an illusory visual percept. Crucially, when the three characters formed a meaningful word, the lack of a concurrent beep suppressed the awareness to a greater extent. A separate experiment replicated the effect on participants who were unable to recognize the words. When the reading was disrupted by reversing the presentation order, the effect disappeared. These findings demonstrate the capacity of our visual system to extract peripheral linguistic information without conscious word recognition.

Forty-Hertz audiovisual stimulation does not have a promoting effect on visual threshold and visual spatial memory.

Previous research has demonstrated that 40-Hz audiovisual stimulation can improve pathological conditions and promote cognitive function in mouse models of Alzheimer's disease. However, limited research has been conducted on humans, and the results have been inconsistent. In our study, we divided participants into an experimental group and a control group to investigate whether 40-Hz stimulation could enhance performance in visual threshold tasks and working memory task. In Experiment 1, we used a light bulb as the stimulus source and found a general practice effect, but no difference between the groups. In Experiment 2, we used a computer screen as the stimulus source and set the stimulation frequency to 48 Hz. In Experiment 3 , we used a computer screen and audio as stimulus sources, simultaneously applying a 40-Hz stimulation to both visual and auditory modalities. Both experiments only revealed the disappearance of practice effects in the 40-Hz (48-Hz) group. Experiment 4 focused on testing visual spatial memory, but did not identify any significant differences between or within groups. In Experiment 5, we tested various visual spatial frequencies; yet again, no significant differences were found. Based on the comprehensive results, we conclude that a 40-Hz stimulation does not have a promoting effect on visual threshold or visual spatial memory.

Extracting probability in the absence of visual awareness.

Extracting statistical regularities from the environment is crucial for survival. It allows us to learn cues for where and when future events will occur. Can we learn these associations even when the cues are not consciously perceived? Can these unconscious processes integrate information over long periods of time? We show that human visual system can track the probability of location contingency between an unconscious prime and a conscious target over a period of time of minutes. In a series of psychophysical experiments, we adopted an exogenous priming paradigm and manipulated the location contingency between a masked prime and a visible target (i.e., how likely the prime location predicted the target location). The prime's invisibility was verified both subjectively and objectively. Although the participants were unaware of both the existence of the prime and the prime-target contingency, our results showed that the probability of location contingency was tracked and manifested in the subsequent priming effect. When participants were first entrained into the fully predictive prime-target probability, they exhibited faster responses to the more predictive location. On the contrary, when no contingency existed between the prime and target initially, participants later showed faster responses to the less predictive location. These results were replicated in two more experiments with increased statistical power and a fine-grained delineation of prime awareness. Together, we report that the human visual system is capable of tracking unconscious probability over a period of time, demonstrating how implicit and uncertain regularity guides behavior.

Neural correlates of individual differences in predicting ambiguous sounds comprehension level.

This study investigated brain activation during auditory processing as a biomarker for the prediction of future perceptual learning performance. Cochlear implant simulated sounds (vocoded sounds) are degraded signals. Participants with normal hearing who were trained with these ambiguous sounds showed varied speech comprehension levels. We discovered that the neuronal signatures from untrained participants forecasted their future ambiguous speech comprehension levels. Participants' brain activations for auditory information processing were measured before (t1) they underwent a five-day vocoded sounds training session. We showed that the pre-training (t1) activities in the inferior frontal gyrus (IFG) correlate with the fifth-day (t2) vocoded sound comprehension performance. To further predict participants' future (t2) performances, we split the participants into two groups (i.e., good and bad learners) based on their fifth-day performance; a linear support vector machine (SVM) was trained to classify (predict) the remaining participants' groups. We found that pre-training (t1) fMRI activities in the bilateral IFG, angular gyrus (AG), and supramarginal gyrus (SMG) showed discriminability between future (t2) good and bad learners. These findings suggest that neural correlates of individual differences in auditory processing can potentially be used to predict participants' future cognition and behaviors.

Detecting spontaneous deception in the brain.

Deception detection can be of great value during the juristic investigation. Although the neural signatures of deception have been widely documented, most prior studies were biased by difficulty levels. That is, deceptive behavior typically required more effort, making deception detection possibly effort detection. Furthermore, no study has examined the generalizability across instructed and spontaneous responses and across participants. To explore these issues, we used a dual-task paradigm, where the difficulty level was balanced between truth-telling and lying, and the instructed and spontaneous truth-telling and lying were collected independently. Using Multivoxel pattern analysis, we were able to decode truth-telling versus lying with a balanced difficulty level. Results showed that the angular gyrus (AG), inferior frontal gyrus (IFG), and postcentral gyrus could differentiate lying from truth-telling. Critically, linear classifiers trained to distinguish instructed truthful and deceptive responses could correctly differentiate spontaneous truthful and deceptive responses in AG and IFG with above-chance accuracy. In addition, with a leave-one-participant-out analysis, multivoxel neural patterns from AG could classify if the left-out participant was lying or not in a trial. These results indicate the commonality of neural responses subserved instructed and spontaneous deceptive behavior as well as the feasibility of cross-participant deception validation.

Subliminal temporal integration of linguistic information under discontinuous flash suppression.

Whether unconscious complex visual information integration occurs over time remains largely unknown and highly controversial. Previous studies have tended to use a combination of strong masking or suppression and a weak stimulus signal (e.g., low luminance), resulting in a low signal-to-noise ratio during unconscious stimulus presentation. To lengthen the stimulus exposure, we introduced intermittent presentation into interocular suppression. This discontinuous suppression allowed us to insert a word during each suppression period and deliver multiple words over time unconsciously. We found that, after participants received the subliminal context, they responded faster to a syntactically incongruent target word in a lexical decision task. We later replicated the finding in a separate experiment where participants exhibited chance performance on locating the subliminal context. These results confirmed that the sentential context was both subjectively and objectively subliminal. Critically, the effect disappeared when the context was disrupted by presenting only partial sentences or sentences with a reversed word order. These control experiments showed that the effect was not merely driven by word-word association but instead required integration over multiple words in the correct order. These findings support the possibility of unconscious high-level, complex information integration.

Microsleep is associated with brain activity patterns unperturbed by auditory inputs.

Microsleeps are brief episodes of arousal level decrease manifested through behavioral signs. Brain activity during microsleep in the presence of external stimulus remains poorly understood. In this study, we sought to understand neural responses to auditory stimulation during microsleep. We gave participants the simple task of listening to audios of different pitches and amplitude modulation frequencies during early afternoon functional MRI scans. We found the following: 1) microsleep was associated with cortical activations in broad motor and sensory regions and deactivations in thalamus, irrespective of auditory stimulation; 2) high and low pitch audios elicited different activity patterns in the auditory cortex during awake but not microsleep state; and 3) during microsleep, spatial activity patterns in broad brain regions were similar regardless of the presence or types of auditory stimulus (i.e., stimulus invariant). These findings show that the brain is highly active during microsleep but the activity patterns across broad regions are unperturbed by auditory inputs.NEW & NOTEWORTHY During deep drowsy states, auditory inputs could induce activations in the auditory cortex, but the activation patterns lose differentiation to high/low pitch stimuli. Instead of random activations, activity patterns across the brain during microsleep appear to be structured and may reflect underlying neurophysiological processes that remain unclear.

Center-surround velocity-based segmentation: Speed, eccentricity, and timing of visual stimuli interact to determine interocular dominance.

We used a novel method to capture the spatial dominance pattern of competing motion fields at rivalry onset. When rivaling velocities were different, the participants reported center-surround segmentation: The slower stimuli often dominated in the center while faster motion persisted along the borders. The size of the central static/slow field scaled with the stimulus size. The central dominance was time-locked to the static stimulus onset but was disrupted if the dynamic stimulus was presented later. We then used the same stimuli as masks in an interocular suppression paradigm. The local suppression strengths were probed with targets at different eccentricities. Consistent with the center-surround segmentation, target speed and location interacted with mask velocities. Specifically, suppression power of the slower masks was nonhomogenous with eccentricity, providing a potential explanation for center-surround velocity-based segmentation. This interaction of speed, eccentricity, and timing has implications for motion processing and interocular suppression. The influence of different masks on which target features get suppressed predicts that some "unconscious effects" are not generalizable across masks and, thus, need to be replicated under various masking conditions.

Cerebral neural correlates of differential melanopic photic stimulation in humans.

Photic stimulation of rods, cones and intrinsically photosensitive melanopsin-containing retinal ganglion cells (ipRGCs) mediates non-visual light responses, including entrainment of circadian rhythms and pupillary light reflex. Unlike visual responses to photic stimulation, the cerebral correlates of non-visual light responses in humans remains elusive. In this study, we used functional magnetic resonance imaging (fMRI) in 14 healthy young participants, to localize cerebral regions which are differentially activated by metameric light that gave rise to different levels of melanopic excitation. Mean blood oxygen-level dependent (BOLD) responses disclosed bilateral activation of the frontal eye fields during exposure to light geared towards melanopsin. Furthermore, multivariate pattern analyses showed distinct bilateral pattern activity in the inferior temporal gyri and the caudate nuclei. Taken together, our findings suggest that melanopsin-based photoreception activates a cerebral network including frontal regions, classically involved in attention and ocular motor responses.

Can a Word Sound Like a Shape Before You Have Seen It? Sound-Shape Mapping Prior to Conscious Awareness.

Nonarbitrary mappings between sound and shape (i.e., the bouba-kiki effect) have been shown across different cultures and early in development; however, the level of processing at which this effect arises remains unclear. Here we show that the mapping occurs prior to conscious awareness of the visual stimuli. Under continuous flash suppression, congruent stimuli (e.g., "kiki" inside an angular shape) broke through to conscious awareness faster than incongruent stimuli. This was true even when we trained people to pair unfamiliar letters with auditory word forms, a result showing that the effect was driven by the phonology, not the visual features, of the letters. Furthermore, visibility thresholds of the shapes decreased when they were preceded by a congruent auditory word form in a masking paradigm. Taken together, our results suggest that sound-shape mapping can occur automatically prior to conscious awareness of visual shapes, and that sensory congruence facilitates conscious awareness of a stimulus being present.

Seeing the sound after visual loss: functional MRI in acquired auditory-visual synesthesia.

Acquired auditory-visual synesthesia (AVS) is a rare neurological sign, in which specific auditory stimulation triggers visual experience. In this study, we used event-related fMRI to explore the brain regions correlated with acquired monocular sound-induced phosphenes, which occurred 2 months after unilateral visual loss due to an ischemic optic neuropathy. During the fMRI session, 1-s pure tones at various pitches were presented to the patient, who was asked to report occurrence of sound-induced phosphenes by pressing one of the two buttons (yes/no). The brain activation during phosphene-experienced trials was contrasted with non-phosphene trials and compared to results obtained in one healthy control subject who underwent the same fMRI protocol. Our results suggest, for the first time, that acquired AVS occurring after visual impairment is associated with bilateral activation of primary and secondary visual cortex, possibly due to cross-wiring between auditory and visual sensory modalities.

Speed-size illusion correlates with retinal-level motion statistics.

It is a common perceptual experience that smaller objects appear to move faster than larger ones when their physical speeds are the same in either the laboratory or daily life. In this study, we show that the speed-size illusion is correlated with retinal image speed distribution bias. The illusion was quantified with a two-alternative, forced choice speed comparison paradigm, and retinal image speed distributions for different image sizes were obtained by simulation. Simulation results show that smaller retinal images tend to have slower projected speed, and the retinal image speed distribution bias correlates with the strength of the speed-size illusion. Furthermore, exposure to a training movie containing unnatural motion statistics tended to modulate the illusion in a way that was consistent with the speed distribution bias. We discuss how the data could be explained by empirical ranking theory, Bayesian theory, and motion adaptation.

Pre-existing brain states predict risky choices.

Rational decision-making models assume that people resolve an economic problem based on its properties and the underlying utility. Here we challenge this view by examining whether pre-stimulus endogenous neuronal fluctuations can bias economic decisions. We recorded subjects' pre-stimulus neural activation patterns with fMRI before presentation and choice between pairs of certain outcomes and risky gambles. Our results indicate that activities in the left nucleus accumbens and medial frontal gyrus can bias subsequent risky decision making, showing that neuronal activities in regions associated with uncertainty and reward processing are involved in biasing subsequent choice selection. This finding challenges theories which propose that choices merely reveal stable underlying distributions of hedonic utility. Endogenous brain states of this sort might originate from a systematic cause or a stochastic type of neural noise, which can be construed as contextual factors that shape people's decision making.

Pre-existing brain states predict aesthetic judgments.

Intuition and an assumption of basic rationality would suggest that people evaluate a stimulus on the basis of its properties and their underlying utility. However, various findings suggest that evaluations often depend not only on what is being evaluated, but also on contextual factors. Here we demonstrate a further departure from normative decision making: Aesthetic evaluations of abstract fractal art by human subjects were predicted from pre-stimulus patterns of BOLD fMRI signals across a distributed network of frontal regions before the stimuli were presented. This predictive power was dissociated from motor biases in favor of pressing a particular button to indicate one's choice. Our findings suggest that endogenous neural signals present before stimulation can bias decisions at multiple levels of representation when evaluating stimuli.

Unconscious cues bias first saccades in a free-saccade task.

Visual-spatial attention can be biased towards salient visual information without visual awareness. It is unclear, however, whether such bias can further influence free-choices such as saccades in a free viewing task. In our experiment, we presented visual cues below awareness threshold immediately before people made free saccades. Our results showed that masked cues could influence the direction and latency of the first free saccade, suggesting that salient visual information can unconsciously influence free actions.

Neural representations of perspectival shapes and attentional effects: Evidence from fMRI and MEG.

Does the human brain represent perspectival shapes, i.e., viewpoint-dependent object shapes, especially in relatively higher-level visual areas such as the lateral occipital cortex? What is the temporal profile of the appearance and disappearance of neural representations of perspectival shapes? And how does attention influence these neural representations? To answer these questions, we employed functional magnetic resonance imaging (fMRI), magnetoencephalography (MEG), and multivariate decoding techniques to investigate spatiotemporal neural representations of perspectival shapes. Participants viewed rotated objects along with the corresponding objective shapes and perspectival shapes (i.e., rotated round, round, and oval) while we measured their brain activities. Our results revealed that shape classifiers trained on the basic shapes (i.e., round and oval) consistently identified neural representations in the lateral occipital cortex corresponding to the perspectival shapes of the viewed objects regardless of attentional manipulations. Additionally, this classification tendency toward the perspectival shapes emerged approximately 200 ms after stimulus presentation. Moreover, attention influenced the spatial dimension as the regions showing the perspectival shape classification tendency propagated from the occipital lobe to the temporal lobe. As for the temporal dimension, attention led to a more robust and enduring classification tendency towards perspectival shapes. In summary, our study outlines a spatiotemporal neural profile for perspectival shapes that suggests a greater degree of perspectival representation than is often acknowledged.

Individual variability in neural representations of mind-wandering.

Mind-wandering is a frequent, daily mental activity, experienced in unique ways in each person. Yet neuroimaging evidence relating mind-wandering to brain activity, for example in the default mode network (DMN), has relied on population-rather than individual-based inferences due to limited within-individual sampling. Here, three densely-sampled individuals each reported hundreds of mind-wandering episodes while undergoing multi-session functional magnetic resonance imaging. We found reliable associations between mind-wandering and DMN activation when estimating brain networks within individuals using precision functional mapping. However, the timing of spontaneous DMN activity relative to subjective reports, and the networks beyond DMN that were activated and deactivated during mind-wandering, were distinct across individuals. Connectome-based predictive modeling further revealed idiosyncratic, whole-brain functional connectivity patterns that consistently predicted mind-wandering within individuals but did not fully generalize across individuals. Predictive models of mind-wandering and attention that were derived from larger-scale neuroimaging datasets largely failed when applied to densely-sampled individuals, further highlighting the need for personalized models. Our work offers novel evidence for both conserved and variable neural representations of self-reported mind-wandering in different individuals. The previously-unrecognized inter-individual variations reported here underscore the broader scientific value and potential clinical utility of idiographic approaches to brain-experience associations.

Individual variability in neural representations of mind-wandering.

Mind-wandering is a frequent, daily mental activity, experienced in unique ways in each person. Yet neuroimaging evidence relating mind-wandering to brain activity, for example in the default mode network (DMN), has relied on population- rather than individual-based inferences owing to limited within-person sampling. Here, three densely sampled individuals each reported hundreds of mind-wandering episodes while undergoing multi-session functional magnetic resonance imaging. We found reliable associations between mind-wandering and DMN activation when estimating brain networks within individuals using precision functional mapping. However, the timing of spontaneous DMN activity relative to subjective reports, and the networks beyond DMN that were activated and deactivated during mind-wandering, were distinct across individuals. Connectome-based predictive modeling further revealed idiosyncratic, whole-brain functional connectivity patterns that consistently predicted mind-wandering within individuals but did not fully generalize across individuals. Predictive models of mind-wandering and attention that were derived from larger-scale neuroimaging datasets largely failed when applied to densely sampled individuals, further highlighting the need for personalized models. Our work offers novel evidence for both conserved and variable neural representations of self-reported mind-wandering in different individuals. The previously unrecognized interindividual variations reported here underscore the broader scientific value and potential clinical utility of idiographic approaches to brain-experience associations.

While everyone experiences that their mind "wanders" throughout daily life, the content and form of inner experience is different in different people. In this study, we found that brain activity representing mind-wandering is different in each person, reflecting unique mental experiences. While people consistently engaged the brain's default mode network (DMN) during mind-wandering, there were inconsistencies in the way that the DMN was engaged and in the other networks throughout the brain that were engaged. Our study highlights that personalized approaches, which require that individuals are sampled more densely than is common in current practice, enable accurate insights into relationships between brain activity and inner experience.

Eye-specific information biases perceived direction of bistable motion.

While eye-of origin information is normally not accessible to observers, processing of visual information within a monocular channel does contribute to our final percept. Here we investigate if visual information is processed more efficiently when it is contained within a monocular channel or across two eyes and how it affects visual perception. Specifically, we used a bistable apparent motion display, a motion quartet, to investigate the role of eye-specific information in determining perceived motion direction. To an observer, this ambiguous display leads to the perception of either horizontal or vertical motion. We attempted to bias perceived direction by presenting separate spatial halves of the motion quartet to each eye. Our results show that observers were more likely to see horizontal motion when top and bottom halves of the quartet was presented to separate eyes. Similarly, when left and right halves were presented dichoptically, observers reported viewing more vertical motion. This change in proportion of observed motion direction indicates that eye specific information is processed more efficiently and can bias overall perception.

Search for patterns of functional specificity in the brain: a nonparametric hierarchical Bayesian model for group fMRI data.

Functional MRI studies have uncovered a number of brain areas that demonstrate highly specific functional patterns. In the case of visual object recognition, small, focal regions have been characterized with selectivity for visual categories such as human faces. In this paper, we develop an algorithm that automatically learns patterns of functional specificity from fMRI data in a group of subjects. The method does not require spatial alignment of functional images from different subjects. The algorithm is based on a generative model that comprises two main layers. At the lower level, we express the functional brain response to each stimulus as a binary activation variable. At the next level, we define a prior over sets of activation variables in all subjects. We use a Hierarchical Dirichlet Process as the prior in order to learn the patterns of functional specificity shared across the group, which we call functional systems, and estimate the number of these systems. Inference based on our model enables automatic discovery and characterization of dominant and consistent functional systems. We apply the method to data from a visual fMRI study comprised of 69 distinct stimulus images. The discovered system activation profiles correspond to selectivity for a number of image categories such as faces, bodies, and scenes. Among systems found by our method, we identify new areas that are deactivated by face stimuli. In empirical comparisons with previously proposed exploratory methods, our results appear superior in capturing the structure in the space of visual categories of stimuli.