The association of targets modulates the search efficiency in multitarget searches.

Previous studies have found that distractors can affect visual search efficiency when associated with the target in a single-target search. However, multitarget searches are frequently necessary in daily life. In the present study, we examined how the association of targets in a multitarget search affected performance when searching for two targets simultaneously (Experiment 1). In addition, we explored whether the association affected switch cost (Experiment 2) and preparation cost (Experiment 3). Participants were required to learn associations between different colors or shapes and then performed feature search and conjunction search tasks. For all experiments, the results of search efficiency showed that for conjunction search, the search efficiency under the associated condition was significantly higher than that under the neutral condition. Similarly, the response times in the associated condition were significantly faster than those in the neutral condition under the conjunction search condition in Experiments 1 and 2. However, in Experiment 3, the response times in the associated condition were longer than those in the neutral condition. These results indicate that the association between targets can improve the efficiency of multitarget searches. Furthermore, associations can reduce the time spent searching for individual targets and the switch cost; however, the preparation cost increases.

Prior Knowledge Uses Prestimulus Alpha Band Oscillations and Persistent Poststimulus Neural Templates for Conscious Perception.

Prior knowledge has a profound impact on the way we perceive the world. However, it remains unclear how the prior knowledge is maintained in our brains and thereby influences the subsequent conscious perception. The Dalmatian dog illusion is a perfect tool to study prior knowledge, where the picture is initially perceived as noise. Once the prior knowledge was introduced, a Dalmatian dog could be consciously seen, and the picture immediately became meaningful. Using pictures with hidden objects as standard stimuli and similar pictures without hidden objects as deviant stimuli, we investigated the neural representation of prior knowledge and its impact on conscious perception in an oddball paradigm using electroencephalogram (EEG) in both male and female human subjects. We found that the neural patterns between the prestimulus alpha band oscillations and poststimulus EEG activity were significantly more similar for the standard stimuli than for the deviant stimuli after prior knowledge was provided. Furthermore, decoding analysis revealed that persistent neural templates were evoked after the introduction of prior knowledge, similar to that evoked in the early stages of visual processing. In conclusion, the current study suggests that prior knowledge uses alpha band oscillations in a multivariate manner in the prestimulus period and induces specific persistent neural templates in the poststimulus period, enabling the conscious perception of the hidden objects.SIGNIFICANCE STATEMENT The visual world we live in is not always optimal. In dark or noisy environments, prior knowledge can help us interpret imperfect sensory signals and enable us to consciously perceive hidden objects. However, we still know very little about how prior knowledge works at the neural level. Using the Dalmatian dog illusion and multivariate methods, we found that prior knowledge uses prestimulus alpha band oscillations to carry information about the hidden object and exerts a persistent influence in the poststimulus period by inducing specific neural templates. Our findings provide a window into the neural underpinnings of prior knowledge and offer new insights into the role of alpha band oscillations and neural templates associated with conscious perception.

The speed and temporal frequency of visual apparent motion modulate auditory duration perception.

In the present study, we investigated how the perception of auditory duration could be modulated by a task-irrelevant, concurrent visual apparent motion, induced by visual bars alternating between left and right sides. Moreover, we examined the influence of the speed and temporal frequency of visual apparent motion on the perception of auditory duration. In each trial, the standard visual stimuli (two vertical bars) were presented sequentially, except that visual apparent motion was included in the fourth stimulus. A tone was presented simultaneously with each visual stimulus, while the fourth tone was presented with varied duration. Participants judged whether the fourth tone lasted longer than the other four tones. In Experiment 1, the speed of visual apparent motion (Fast vs. Slow) was manipulated by changing the interval between two bars. The mean point of subjective equality (PSE) in the Slow apparent motion condition was larger than that in the Static condition. Moreover, participants tended to overestimate the duration only in the Static condition, i.e., time dilation effect, which disappeared under apparent motion conditions. In Experiment 2, in addition to speed, we controlled the temporal frequency of apparent motion by manipulating the number of bars, generating four conditions of visual apparent motion (Physical-fast, Perceived-fast, Perceived-slow, vs. Static). The mean PSE was significantly smaller in the Physical-fast condition than in the Static and Perceived-slow conditions. Moreover, we found a time compression effect in both the Perceived-slow and Static conditions but not in the Perceived-fast and Physical-fast conditions. These results suggest that the auditory duration could be modulated by the concurrent, contextual visual apparent motion, and both the speed and temporal frequency of the task-irrelevant visual apparent motion contribute to the bias in perceiving the auditory duration.

Working memory load modulates the processing of audiovisual distractors: A behavioral and event-related potentials study.

The interplay between different modalities can help to perceive stimuli more effectively. However, very few studies have focused on how multisensory distractors affect task performance. By adopting behavioral and event-related potentials (ERPs) techniques, the present study examined whether multisensory audiovisual distractors could attract attention more effectively than unisensory distractors. Moreover, we explored whether such a process was modulated by working memory load. Across three experiments, n-back tasks (1-back and 2-back) were adopted with peripheral auditory, visual, or audiovisual distractors. Visual and auditory distractors were white discs and pure tones (Experiments 1 and 2), pictures and sounds of animals (Experiment 3), respectively. Behavioral results in Experiment 1 showed a significant interference effect under high working memory load but not under low load condition. The responses to central letters with audiovisual distractors were significantly slower than those to letters without distractors, while no significant difference was found between unisensory distractor and without distractor conditions. Similarly, ERP results in Experiments 2 and 3 showed that there existed an integration only under high load condition. That is, an early integration for simple audiovisual distractors (240-340 ms) and a late integration for complex audiovisual distractors (440-600 ms). These findings suggest that multisensory distractors can be integrated and effectively attract attention away from the main task, i.e., interference effect. Moreover, this effect is pronounced only under high working memory load condition.

Does auditory numerosity and non-numerical magnitude affect visual non-symbolic numerical representation?

In the present study, we investigated the effects of auditory numerosity and magnitude (loudness) on visual numerosity processing. Participants compared numerosities of two sequential dot arrays. The second dot array was paired with a tone array that was independent of visual comparison. The numerosity (One-tone vs. Multiple-tone) and the non-numerical magnitude of tones (loudness) were manipulated in Experiments 1 and 2, respectively. In Experiment 1, participants' inverse efficiency score (IES), that is, the quotient between response time and accuracy, was significantly smaller in the One-tone and Multiple-tone conditions than that in the No-tone condition, and linear trend analyses showed that the IES decreased with the number of tones. In Experiment 2, the IES in the Loud-tone condition was significantly smaller than that in the No-tone condition, and the IES decreased as the loudness of the tones increased. In Experiment 3, both auditory numerosity and magnitude were manipulated. For soft tones, the IES was smaller in the Multiple-tone condition than in the One-tone condition, whereas no significant difference was found between two conditions in loud tones. In sum, these findings suggest that the visual numerical representation can be spontaneously affected by the numerosity and non-numerical magnitude of stimuli from another modality.

Does Eye Gaze Uniquely Trigger Spatial Orienting to Socially Relevant Information? A Behavioral and ERP Study.

Using behavioral and event-related potential (ERP) measures, the present study examined whether eye gaze triggers a unique form of attentional orienting toward threat-relevant targets. A threatening or neutral target was presented after a non-predictive gaze or an arrow cue. In Experiment 1, reaction times indicated that eye gaze and arrow cues triggered different attention orienting towards threatening targets, which was confirmed by target-elicited P3b latency in Experiment 2. Specifically, for targets preceded by arrow and gaze cues, P3b peak latency was shorter for neutral targets than threatening targets. However, the latency differences were significantly smaller for gaze cues than for arrow cues. Moreover, target-elicited N2 amplitude indicated a significantly stronger cue validity effect of eye gaze than that of arrows. These findings suggest that eye gaze uniquely triggers spatial attention orienting to socially threatening information.

Tinnitus Affects Endogenous But Not Exogenous Auditory Attention Orienting.

PURPOSE: Previous studies have demonstrated that people with tinnitus show attention dysfunctions. In this study, we investigated the influence of tinnitus on attention orienting, especially whether the ability of attention orienting could be modulated by the degree of tinnitus. METHOD: Fifty-nine and 54 unilateral tinnitus participants were included in Experiment 1 and Experiment 2, respectively. All participants reported subjective tinnitus for at least 3 months and were divided into a mild tinnitus group (Tinnitus Handicap Inventory [THI] < 37) or a moderate to severe tinnitus group (THI ≥ 37) according to the THI score. An auditory exogenous attention task and an auditory endogenous attention task were adopted. In the exogenous task, a target sound following a cue sound was presented on either the left or right side. Participants were required to discriminate whether the target was pure tone or white noise. In the endogenous task, participants were required to pay attention to the stimuli on one side and judge the pitch of a target sound. Mixed-design analyses of variance were conducted for the mean reaction times and accuracy across the experimental conditions. RESULTS: Our results showed that in the endogenous attention task, compared with the mild tinnitus group, moderate to severe tinnitus participants had better performance for stimuli presented on the tinnitus side but not on the nontinnitus side. In contrast, in the exogenous attention task, no differences were found between mild and moderate to severe tinnitus groups. CONCLUSION: The results suggest that the degree of tinnitus influences the performance of auditory endogenous attention but not auditory exogenous attention orienting.

Does gaze direction of fearful faces facilitate the processing of threat? An ERP study of spatial precuing effects.

Eye gaze is very important for attentional orienting in social life. By adopting the event-related potential (ERP) technique, we explored whether attentional orienting of eye gaze is modulated by emotional congruency between facial expressions and the targets in a spatial cuing task. Faces with different emotional expressions (fearful/angry/happy/neutral) directing their eye gaze to the left or right were used as cues, indicating the possible location of subsequent targets. Targets were line drawings of animals, which could be either threatening or neutral. Participants indicated by choice responses whether the animal would fit inside a shoebox in real life or not. Reaction times to targets were faster after valid compared with invalid cues, showing the typical eye gaze cuing effect. Analyses of the late positive potential (LPP) elicited by targets revealed a significant modulation of the gaze cuing effect by emotional congruency. Threatening targets elicited larger LPPs when validly cued by gaze in faces with negative (fearful and angry) expressions. Similarly, neutral targets showed larger LPPs when validly cued by faces with neutral expressions. Such effects were not present after happy face cues. Source localization in the LPP time window revealed that for threatening targets, the activity of right medial frontal gyrus could be related to a larger gaze-orienting effect for the fearful than the angry condition. Our findings provide electrophysiological evidence for the modulation of gaze cuing effects by emotional congruency.

Audio-Visual Causality and Stimulus Reliability Affect Audio-Visual Synchrony Perception.

People can discriminate the synchrony between audio-visual scenes. However, the sensitivity of audio-visual synchrony perception can be affected by many factors. Using a simultaneity judgment task, the present study investigated whether the synchrony perception of complex audio-visual stimuli was affected by audio-visual causality and stimulus reliability. In Experiment 1, the results showed that audio-visual causality could increase one's sensitivity to audio-visual onset asynchrony (AVOA) of both action stimuli and speech stimuli. Moreover, participants were more tolerant of AVOA of speech stimuli than that of action stimuli in the high causality condition, whereas no significant difference between these two kinds of stimuli was found in the low causality condition. In Experiment 2, the speech stimuli were manipulated with either high or low stimulus reliability. The results revealed a significant interaction between audio-visual causality and stimulus reliability. Under the low causality condition, the percentage of "synchronous" responses of audio-visual intact stimuli was significantly higher than that of visual_intact/auditory_blurred stimuli and audio-visual blurred stimuli. In contrast, no significant difference among all levels of stimulus reliability was observed under the high causality condition. Our study supported the synergistic effect of top-down processing and bottom-up processing in audio-visual synchrony perception.

Auditory Selective Attention Hindered by Visual Stimulus in Prelingually Deaf Children With Cochlear Implants.

OBJECTIVE: To compare the influence of visual distractors on the performance of auditory selective attention between prelingually deaf children with a CI (cochlear implant) and children with normal-hearing. DESIGN: Twenty-two patients who had a cochlear implant device (10 males and 12 females, aged 6.64 ±â€Š0.99 yrs) and 16 normal-hearing children (6 males and 10 females, aged 6.09 ±â€Š0.51 yrs) were recruited. Half of the auditory stimuli were presented together with visual stimuli, and participants were required to complete an auditory identification task. Reaction times and discriminability (d') for these two groups were recorded and calculated. RESULTS: The normal-hearing group had shorter mean reaction times than the CI group in detecting auditory targets. With visual distraction, the d' of the normal-hearing group was significantly better than that of CI group (t = 2.649, p = 0.012), while no statistical significance was found between the two groups without visual distraction (t = 0.693, p = 0.493). CONCLUSION: Enhanced processing of visual stimuli interferes with auditory perception in CI users by occupying the capacity-limited attention.

Introduction to the Special Issue on Multisensory Research Forum (IMRF 2016, Suzhou).

This editorial introduction briefly summarized the general information and keynote topics of 17th International Multisensory Research Forum (IMRF 2016), which was held in Suzhou, China on June 15-18, 2016. In this IMRF2016 Multisensory Research special issue, seven papers have been collected. Some topics represent mainstream, traditional studies, while others show a range of multidisciplinary approaches. We categorized the main topics in this issue to four aspects: expertise in multisensory timing, neuronal signatures underlying cross-modal correspondence, coupling between perception and action, and lastly, a revisit of the principles for multisensory segregation and integration.

Positive Emotion Facilitates Cognitive Flexibility: An fMRI Study.

Cognitive flexibility is the ability to switch rapidly between multiple goals. By using a task-switching paradigm, the present study investigated how positive emotion affected cognitive flexibility and the underlying neural mechanisms. After viewing pictures of different emotional valence (positive, negative, or neutral), participants discriminated whether a target digit in a specific color was odd or even. After a series of trials, the color of target stimuli was changed, i.e., the switch condition. Switch costs were measured by the increase of reaction times (RTs) in the switch trials compared to those in the repeat trials. Behavior results indicated that switch costs significantly decreased in the positive emotional condition, and increased in the negative emotional condition, compared with those in the neutral condition. Imaging data revealed enhanced activation in the dorsal anterior cingulate cortex (dACC) in switch trials than those in repeat trials. Moreover, the interaction between emotion (positive, negative, neutral) and trial type (repeat vs. switch) was significant. For switch trials, the activation of dACC decreased significantly in the positive condition, while increased significantly in the negative condition compared to neutral condition. By contrast, for repeat trials, no significant difference was observed for the activation of dACC among three emotional conditions. Our results showed that positive emotions could increase the cognitive flexibility and reduce the conflict by decreasing the activation of dACC.

Resilience mediated the association between acculturation and psychological growth in college students from Hong Kong to Guangzhou, China.

BACKGROUND: The number of Hong Kong citizens living in mainland China is increasing. The process of acculturation may create opportunities for psychological growth. AIMS: This study aimed at examining whether resilience mediated the effects of acculturation on psychological growth in college students from Hong Kong to Guangzhou. METHOD: In this cross-sectional survey, 164 college students in Guangzhou who were Hong Kong permanent residents (female: 46%, age: 21.09 ± 1.50) joined the study. RESULTS: The integration group reported more psychological growth than the assimilation, separation, and marginalization groups. Resilience partially mediated the effect of integration on psychological growth and fully mediated the effect of marginalization on psychological growth. CONCLUSIONS: Resilience represents one of the mechanisms to explain beneficial effects of integration on psychological growth in college students from Hong Kong to Guangzhou. Theoretical considerations to rethink the acculturation model, implications for developing intervention programs, and recommendations for future research are discussed.

Temporal and spectral dynamics underlying cognitive control modulated by task-irrelevant stimulus-response learning.

Behavioral and recent neuroimaging findings have shown reversal of interference effects due to manipulating proportion congruency (PC), which suggests that task-irrelevant stimulus-response (S-R) associations are strengthened and applied to predict responses. However, it is unclear how the strengthened S-R associations are represented and applied in the brain. We investigated with a between-subjects PC paradigm of the Hedge and Marsh task using electroencephalography (EEG). The behavioral results showed the reversal of the conflict effects, suggesting that task-irrelevant S-R associations were strengthened and used to prepare responses. The EEG results revealed the PC-related reversal of the conflict effects in the frontocentral N2 and parietal P3b amplitudes. Time-frequency analyses showed more pronounced PC-related reversal of the conflict effects in theta band (4-8 Hz) activity in frontocentral sites. These results suggest that the strengthened S-R associations due to PC manipulation modulated cognitive control. Importantly, the amplitude of lateralized readiness potential was higher in the high-PC condition than in the low-PC condition, suggesting that the strengthened short-term-memory spatial S-R associations that modulated cognitive control were applied similarly to long-term-memory spatial S-R associations.

Odors Bias Time Perception in Visual and Auditory Modalities.

Previous studies have shown that emotional states alter our perception of time. However, attention, which is modulated by a number of factors, such as emotional events, also influences time perception. To exclude potential attentional effects associated with emotional events, various types of odors (inducing different levels of emotional arousal) were used to explore whether olfactory events modulated time perception differently in visual and auditory modalities. Participants were shown either a visual dot or heard a continuous tone for 1000 or 4000 ms while they were exposed to odors of jasmine, lavender, or garlic. Participants then reproduced the temporal durations of the preceding visual or auditory stimuli by pressing the spacebar twice. Their reproduced durations were compared to those in the control condition (without odor). The results showed that participants produced significantly longer time intervals in the lavender condition than in the jasmine or garlic conditions. The overall influence of odor on time perception was equivalent for both visual and auditory modalities. The analysis of the interaction effect showed that participants produced longer durations than the actual duration in the short interval condition, but they produced shorter durations in the long interval condition. The effect sizes were larger for the auditory modality than those for the visual modality. Moreover, by comparing performance across the initial and the final blocks of the experiment, we found odor adaptation effects were mainly manifested as longer reproductions for the short time interval later in the adaptation phase, and there was a larger effect size in the auditory modality. In summary, the present results indicate that odors imposed differential impacts on reproduced time durations, and they were constrained by different sensory modalities, valence of the emotional events, and target durations. Biases in time perception could be accounted for by a framework of attentional deployment between the inducers (odors) and emotionally neutral stimuli (visual dots and sound beeps).

Independence of the completion effect from the noncompletion effect in illusory contour perception.

Spatially separated object information can be effortlessly completed in the visual system, as demonstrated by the well-known Kanizsa-type illusory contours. The perception of illusory contours is closely associated with the spatial configuration of contour fragments, leading to the long-lasting difficulty in distinguishing the effect of the completion process that interpolates the contour fragments from the effect of the noncompletion process that analyzes the contour fragments. However, a close relationship does not necessarily imply nonindependence, e.g., two people may show similar behaviors in one situation but may not in another situation. Inspired by this simple common sense, we conducted a contour discrimination task (i.e., discriminating between the interpolated contours) and a fragment discrimination task (i.e., discriminating between the physically-specified contour fragments) for Kanizsa squares and Kanizsa circles. The performance difference between the contour and fragment discrimination tasks was much larger for Kanizsa circles than for Kanizsa squares. This independence of the completion effect--as indicated by the performance in the contour task--from the noncompletion effect--as indicated by the performance in the fragment task--provides new insights into the understanding of the mechanism of visual completion.

Interaction between spatial inhibition of return (IOR) and executive control in three-dimensional space.

It has been well documented how spatial inhibition of return (IOR) interacts with executive functions in a two-dimensional plane, i.e., significantly decreased interference at the cued (inhibited) compared to the uncued location. It remains unknown, however, how spatial IOR interacts with executive functions along the depth dimension of the real 3D world. Here, we adapted the Posner spatial-cuing paradigm into a virtual 3D world. The location-based IOR was orthogonally combined with the flanker effect: The target and its flanker could appear at either the cued or the uncued location in a closer or farther depth plane. Moreover, the flanker effect was differentiated into the pre-response and response levels, and the flankers could appear in the either same (Experiment 1) or different (Experiment 2) depth plane from the target. A simple detection task was also adopted to test the pure effect of how visuospatial attention is prevented from returning previously attended location along the depth dimension. The results showed that there existed significant IOR effects only when target was presented in the farther depth plane. Moreover, significantly reversed response-level conflicts were observed at the cued location in the farther depth plane, indicating that spatial IOR toward the farther depth plane was more than a simple effect of attentional orienting. Rather, the inhibitory tagging mechanism may take place. In addition, orienting to the close depth plane resulted in either a facilitatory or null effect. Accordingly, only the pre-response-level conflict was modulated by attentional orienting to the closer depth plane.

Enhanced visual dominance in far space.

The Colavita effect refers to the phenomenon that people do not respond to an auditory stimulus in most cases when a visual stimulus is simultaneously presented. Although the Colavita effect remains robust irrespective of many factors, little is known concerning how the visual dominance varies as a function of the depth of sensory inputs. In the present study, visual and auditory stimuli were presented either in the same (in Experiment 1) or in the different spatial distances (in Experiment 2). Participants were asked to make speeded responses to unimodal auditory, unimodal visual, or bimodal audiovisual stimuli. In the incorrectly responded bimodal trials, the error trials in which responses were made only to the visual component were compared with the trials in which responses were made only to the auditory component. In the correctly responded bimodal trials, the trials in which participants responded first to the visual component were compared with the trials in which participants responded first to the auditory component. Analysis on the incorrect and correct bimodal trials both indicated significant visual dominance effects. More importantly, the size of the visual dominance effect was significantly enhanced as long as the visual stimuli were presented in far space irrespective of whether the auditory stimuli were presented in near or far space. Our results thus, for the first time, revealed that the visual dominance effect changed along the depth dimension of space. Taken together, the present results shed lights on how the allocation of attentional resources along the depth dimension of space biases the process of multisensory competition.

Own-race faces capture attention faster than other-race faces: evidence from response time and the N2pc.

Studies have shown that people are better at recognizing human faces from their own-race than from other-races, an effect often termed the Own-Race Advantage. The current study investigates whether there is an Own-Race Advantage in attention and its neural correlates. Participants were asked to search for a human face among animal faces. Experiment 1 showed a classic Own-Race Advantage in response time both for Chinese and Black South African participants. Using event-related potentials (ERPs), Experiment 2 showed a similar Own-Race Advantage in response time for both upright faces and inverted faces. Moreover, the latency of N2pc for own-race faces was earlier than that for other-race faces. These results suggested that own-race faces capture attention more efficiently than other-race faces.

Dissociable identity- and modality-specific neural representations as revealed by cross-modal nonspatial inhibition of return.

There are ongoing debates on whether object concepts are coded as supramodal identity-based or modality-specific representations in the human brain. In this fMRI study, we adopted a cross-modal "prime-neutral cue-target" semantic priming paradigm, in which the prime-target relationship was manipulated along both the identity and the modality dimensions. The prime and the target could refer to either the same or different semantic identities, and could be delivered via either the same or different sensory modalities. By calculating the main effects and interactions of this 2 (identity cue validity: "Identity_Cued" vs. "Identity_Uncued") × 2 (modality cue validity: "Modality_Cued" vs. "Modality_Uncued") factorial design, we aimed at dissociating three neural networks involved in creating novel identity-specific representations independent of sensory modality, in creating modality-specific representations independent of semantic identity, and in evaluating changes of an object along both the identity and the modality dimensions, respectively. Our results suggested that bilateral lateral occipital cortex was involved in creating a new supramodal semantic representation irrespective of the input modality, left dorsal premotor cortex, and left intraparietal sulcus were involved in creating a new modality-specific representation irrespective of its semantic identity, and bilateral superior temporal sulcus was involved in creating a representation when the identity and modality properties were both cued or both uncued. In addition, right inferior frontal gyrus showed enhanced neural activity only when both the identity and the modality of the target were new, indicating its functional role in novelty detection.