Spatial frequency impacts perceptual and attentional ERP components across cultures.

Culture impacts visual perception in several ways.To identify stages of perceptual processing that differ between cultures, we usedelectroencephalography measures of perceptual and attentional responses to simple visual stimuli.Gabor patches of higher or lower spatialfrequencywere presented at high contrast to 25 American and 31 East Asian participants while they were watching for the onset of aninfrequent, oddball stimulus. Region of interest and mass univariate analyses assessed how cultural background and stimuli spatial frequency affected the visual evoked response potentials. Across both groups, the Gabor of lower spatial frequency produced stronger evoked response potentials in the anterior N1 and P3 than did the higher frequency Gabor. The mass univariate analyses also revealed effects of spatial frequency, including a frontal negativity around 150 ms and a widespread posterior positivity around 300 ms. The effects of spatial frequency generally differed little across cultures; although there was some evidence for cultural differences in the P3 response to different frequencies at the Pz electrode, this effect did not emerge in the mass univariate analyses. We discuss these results in relation to those from previous studies, and explore the potential advantages of mass univariate analyses for cultural neuroscience.

Costs and benefits of audiovisual interactions.

A strong temporal correlation promotes integration of concurrent sensory signals, either within a single sensory modality, or from different modalities. Although the benefits of such integration are well known, far less attention has been given to possible costs incurred when concurrent sensory signals are uncorrelated. In two experiments, subjects categorized the rate at which a visual object modulated in size, while they also tried to ignore a concurrent task-irrelevant broadband sound. Overall, the experiments showed that (i) losses in accuracy from mismatched auditory and visual rates were larger than gains from matched rates and (ii) mismatched auditory and visual rates slowed responses more than they were sped up when rates matched. Experiment One showed that audiovisual interaction varied with the difference between the visual modulation rate and the modulation rate of a concurrent auditory stimulus. Experiment Two showed that audiovisual interaction depended upon the strength of the task-irrelevant auditory modulation. Although our stimuli involved abstract, low-dimensional stimuli, not speech, the effects we observed parallel key findings on interference in multi-speaker settings.

Decision-Making and Multisensory Combination Under Time Stress.

Using a video game platform, we examined how vision-based decision making was affected by a concurrent, potentially conflicting auditory stimulus. Electroencephalographic responses showed that by 150 milliseconds of stimulus onset, the brain had detected the conflict between visual and auditory stimuli. Systematically reducing the intertrial interval (ITI), which subjects described as stressful, undermined decision making. Subjects' arterial pulse variance decreased along with ITI, signaling increased parasympathetic influence on the heart. When successive trials required a shift in processing mode, short ITIs significantly boosted one trial's influence on the next, suggesting that stress reduces cognitive flexibility. Finally, our study demonstrates the heart's and the brain's important influence on decision making.

Catching Audiovisual Interactions With a First-Person Fisherman Video Game.

The human brain is excellent at integrating information from different sources across multiple sensory modalities. To examine one particularly important form of multisensory interaction, we manipulated the temporal correlation between visual and auditory stimuli in a first-person fisherman video game. Subjects saw rapidly swimming fish whose size oscillated, either at 6 or 8 Hz. Subjects categorized each fish according to its rate of size oscillation, while trying to ignore a concurrent broadband sound seemingly emitted by the fish. In three experiments, categorization was faster and more accurate when the rate at which a fish oscillated in size matched the rate at which the accompanying, task-irrelevant sound was amplitude modulated. Control conditions showed that the difference between responses to matched and mismatched audiovisual signals reflected a performance gain in the matched condition, rather than a cost from the mismatched condition. The performance advantage with matched audiovisual signals was remarkably robust over changes in task demands between experiments. Performance with matched or unmatched audiovisual signals improved over successive trials at about the same rate, emblematic of perceptual learning in which visual oscillation rate becomes more discriminable with experience. Finally, analysis at the level of individual subjects' performance pointed to differences in the rates at which subjects can extract information from audiovisual stimuli.

Bi-directional audiovisual influences on temporal modulation discrimination.

Cross-modal interactions of auditory and visual temporal modulation were examined in a game-like experimental framework. Participants observed an audiovisual stimulus (an animated, sound-emitting fish) whose sound intensity and/or visual size oscillated sinusoidally at either 6 or 7 Hz. Participants made speeded judgments about the modulation rate in either the auditory or visual modality while doing their best to ignore information from the other modality. Modulation rate in the task-irrelevant modality matched the modulation rate in the task-relevant modality (congruent conditions), was at the other rate (incongruent conditions), or had no modulation (unmodulated conditions). Both performance accuracy and parameter estimates from drift-diffusion decision modeling indicated that (1) the presence of temporal modulation in both modalities, regardless of whether modulations were matched or mismatched in rate, resulted in audiovisual interactions; (2) congruence in audiovisual temporal modulation resulted in more reliable information processing; and (3) the effects of congruence appeared to be stronger when judging visual modulation rates (i.e., audition influencing vision), than when judging auditory modulation rates (i.e., vision influencing audition). The results demonstrate that audiovisual interactions from temporal modulations are bi-directional in nature, but with potential asymmetries in the size of the effect in each direction.

Online Modulation of Selective Attention is not Impaired in Healthy Aging.

Background/Study Context: Reduced processing speed pervades a great many aspects of human aging and cognition. However, little is known about one aspect of cognitive aging in which speed is of the essence, namely, the speed with which older adults can deploy attention in response to a cue. METHODS: The authors compared rapid temporal modulation of cued visual attention in younger (Mage = 22.3 years) and older (Mage = 68.9 years) adults. On each trial of a short-term memory task, a cue identified which of two briefly presented stimuli was task relevant and which one should be ignored. After a short delay, subjects demonstrated recall by reproducing from memory the task-relevant stimulus. This produced estimates of (i) accuracy with which the task-relevant stimulus was recalled, (ii) the influence of stimuli encountered on previous trials (a prototype effect), and (iii) the influence of the trial's task-irrelevant stimulus. RESULTS: For both groups, errors in recall were considerably smaller when selective attention was cued before rather than after presentation of the stimuli. Both groups showed serial position effects to the same degree, and both seemed equally adept at exploiting the stimuli encountered on previous trials as a means of supplementing recall accuracy on the current trial. CONCLUSION: Younger and older subjects may not differ reliably in capacity for cue-directed temporal modulation of selective attention, or in ability to draw on previously seen stimuli as memory support.

Obligatory and adaptive averaging in visual short-term memory.

Visual memory can draw upon averaged perceptual representations, a dependence that could be both adaptive and obligatory. In support of this idea, we review a wide range of evidence, including findings from our own lab. This evidence shows that time- and space-averaged memory representations influence detection and recognition responses, and do so without instruction to compute or report an average. Some of the work reviewed exploits fine-grained measures of retrieval from visual short-term memory to closely track the influence of stored averages on recall and recognition of briefly presented visual textures. Results show that reliance on perceptual averages is greatest when memory resources are taxed or when subjects are uncertain about the fidelity of their memory representation. We relate these findings to models of how summary statistics impact visual short-term memory, and discuss a neural signature for contexts in which perceptual averaging exerts maximal influence.

Memory and learning with rapid audiovisual sequences.

We examined short-term memory for sequences of visual stimuli embedded in varying multisensory contexts. In two experiments, subjects judged the structure of the visual sequences while disregarding concurrent, but task-irrelevant auditory sequences. Stimuli were eight-item sequences in which varying luminances and frequencies were presented concurrently and rapidly (at 8 Hz). Subjects judged whether the final four items in a visual sequence identically replicated the first four items. Luminances and frequencies in each sequence were either perceptually correlated (Congruent) or were unrelated to one another (Incongruent). Experiment 1 showed that, despite encouragement to ignore the auditory stream, subjects' categorization of visual sequences was strongly influenced by the accompanying auditory sequences. Moreover, this influence tracked the similarity between a stimulus's separate audio and visual sequences, demonstrating that task-irrelevant auditory sequences underwent a considerable degree of processing. Using a variant of Hebb's repetition design, Experiment 2 compared musically trained subjects and subjects who had little or no musical training on the same task as used in Experiment 1. Test sequences included some that intermittently and randomly recurred, which produced better performance than sequences that were generated anew for each trial. The auditory component of a recurring audiovisual sequence influenced musically trained subjects more than it did other subjects. This result demonstrates that stimulus-selective, task-irrelevant learning of sequences can occur even when such learning is an incidental by-product of the task being performed.

Attention-modulated alpha-band oscillations protect against intrusion of irrelevant information.

Combining high-density scalp EEG recordings with a sensitive analog measure of STM's fidelity, we characterized the temporal dynamics of intentional ignoring and related those dynamics to the intrusion of task-irrelevant information. On each trial of the task, two study Gabors were briefly presented in succession. A green or red disc preceding each Gabor signified whether that Gabor should be remembered or ignored, respectively. With cue-stimulus intervals of 300, 600, or 900 msec presented in separate sessions, we found that the onset of posterior, prestimulus alpha oscillations varied with the length of the interval. Although stimulus onset time was entirely predictable, the longer the cue-stimulus interval, the earlier the increase in prestimulus alpha power. However, the alpha-band modulation was not simply locked to the cue offset. The temporal envelopes of posterior alpha-band modulation were strikingly similar for both cued attending and cued ignoring and differed only in magnitude. This similarity suggests that cued attending includes suppression of task-irrelevant, spatial processing. Supporting the view that alpha-band oscillations represent inhibition, our graded measure of recall revealed that, when the stimulus to be ignored appears second in the sequence, peristimulus alpha power predicted the degree to which that irrelevant stimulus distorted subsequent recall of the stimulus that was to be remembered. These results demonstrate that timely deployment of attention-related alpha-band oscillations can aid STM by filtering out task-irrelevant information.

Paying attention to attention in recognition memory: insights from models and electrophysiology.

Reliance on remembered facts or events requires memory for their sources, that is, the contexts in which those facts or events were embedded. Understanding of source retrieval has been stymied by the fact that uncontrolled fluctuations of attention during encoding can cloud results of key importance to theoretical development. To address this issue, we combined electrophysiology (high-density electroencephalogram, EEG, recordings) with computational modeling of behavioral results. We manipulated subjects' attention to an auditory attribute, whether the source of individual study words was a male or female speaker. Posterior alpha-band (8-14 Hz) power in subjects' EEG increased after a cue to ignore the voice of the person who was about to speak. Receiver-operating-characteristic analysis validated our interpretation of oscillatory dynamics as a marker of attention to source information. With attention under experimental control, computational modeling showed unequivocally that memory for source (male or female speaker) reflected a continuous signal detection process rather than a threshold recollection process.

Keep your finger on the pulse: Better rate perception and gap detection with vibrotactile compared to visual stimuli.

Important characteristics of the environment can be represented in the temporal pattern of sensory stimulation. In two experiments, we compared accuracy of temporal processing by different modalities. Experiment 1 examined binary categorization of rate for visual (V) or vibrotactile (T) stimulus pulses presented at either 4 or 6 Hz. Inter-pulse intervals were either constant or variable, perturbed by random Gaussian variates. Subjects categorized the rate of T pulse sequences more accurately than V sequences. In V conditions only, subjects disproportionately tended to mis-categorize 4-Hz pulse rates, for all but the most variable sequences. In Experiment 2, we compared gap detection thresholds across modalities, using the same V and T pulses from Experiment 1, as well as with bimodal (VT) pulses. Visual gap detection thresholds were larger (3[Formula: see text]) than tactile thresholds. Additionally, performance with VT stimuli seemed to be nearly completely dominated by their T components. Together, these results suggest (i) that vibrotactile temporal acuity surpasses visual temporal acuity, and (ii) that vibrotactile stimulation has considerable, untapped potential to convey temporal information like that needed for eyes-free alerting signals.

Audiovisual interaction with rate-varying signals.

A task-irrelevant, amplitude-modulating sound influences perception of a size-modulating visual stimulus. To probe the limits of this audiovisual interaction we vary the second temporal derivative of object size and of sound amplitude. In the study's first phase subjects see a visual stimulus size-modulating with f ″ ( x ) > 0, 0, or <0, and judge each one's rate as increasing, constant, or decreasing. Visual stimuli are accompanied by a steady, non-modulated auditory stimulus. The novel combination of multiple stimuli and multi-alternative responses allows subjects' similarity space to be estimated from the stimulus-response confusion matrix. In the study's second phase, rate-varying visual stimuli are presented in concert with auditory stimuli whose second derivative also varied. Subjects identified each visual stimuli as one of the three types, while trying to ignore the accompanying sound. Unlike some previous results with f ″ ( x ) fixed at 0, performance benefits relatively little when visual and auditory stimuli share the same directional change in modulation. However, performance does drop when visual and auditory stimului differ in their directions of rate change. Our task's computational demands may make it particularly vulnerable to the effects of a dynamic task-irrelevant stimulus.

Feeling rushed? Perceived time pressure impacts executive function and stress.

Executive function (EF) is critical to everyday life, but it can be undermined by adverse psychological states like stress and negative affect. For example, inadequate time to perform a task is a common stressor that can disrupt EF. Although the impact of actual time pressure on EF has been established, little is known about how self-generated, perceived time pressure (PTP) affects EF in the absence of objective time limits. We chose Eriksen's Flanker task as an index of cognitive inhibition, a key component of EF, and we varied the interval between successive trials, the inter-trial interval (ITI), to proxy PTP. This manipulation strongly impacted task performance: shrinking the ITI to increase PTP diminished cognitive inhibition and increased both stress and negative affect. Subsequently lengthening the ITI to decrease PTP reversed nearly all of these effects, except stress, which persisted. Multilevel linear regression modeling revealed that ITI and stress predicted inhibition, and exploratory mediation modeling suggested that stress mediates the relationship between ITI and inhibition. These findings validate perceived time pressure as an empirical stressor and demonstrate EF's sensitivity to changes in PTP.

Attention protects the fidelity of visual memory: behavioral and electrophysiological evidence.

Recall from visual memory is vulnerable to the influence of task-irrelevant information, including the remembered, prototypical value of stimuli seen previously. Wilken and Ma (2004) proposed that this distortion of recall was actually adaptive, with the task-irrelevant information compensating for imperfections of memory. We tested their proposal by using trial-by-trial oscillations in the electroencephalogram's alpha band (8-14 Hz) collected from human subjects as a marker for the strength of visual attention. Subjects' recall of stimulus spatial frequency showed a systematic error, namely a shift toward the prototypical value of previously seen stimuli. The magnitude of this prototype effect was strongly related to the amplitude of alpha band oscillations recorded at posterior sensor locations during the first 100 ms after stimulus onset. Our results support the hypothesis that the prototype effect is compensatory for imprecision in memory. Moreover, the attentional modulation of alpha activity during the encoding of the target stimulus is consistent with the view that attention sharpens the neural responses that are elicited by the task-relevant stimulus.

Eye movements and imitation learning: intentional disruption of expectation.

Over repeated viewings of motion along a quasi-random path, ability to reproduce that path from memory improves. To assess the role of expectations and sequence context on such learning, subjects eye movements were measured while trajectories were viewed for subsequent reproduction. As a sequence of motions was repeated, subjects' eye movements became anticipatory, leading the stimulus' motions. To investigate how prediction errors affected eye movements and imitation learning, we injected an occasional deviant motion into a well-learned stimulus sequence, violating subjects' expectation about the motion that would be seen. This unexpected direction of motion in the stimulus sequence did not impair reproduction of the sequence. The externally induced prediction errors promoted one-shot learning: During the very next stimulus presentation, their eye movements showed that subjects now expected the new sequence item to reappear. A second experiment showed that an associative mismatch can facilitate accurate reproduction of an unexpected stimulus. After a deviant sequence item was presented, imitation accuracy for sequences that contained the deviant direction of motion was reduced relative to sequences that restored the original direction of motions. These findings demonstrate that in the context of a familiar sequence, unexpected events can play an important role in learning the sequence.

Perceptual timing precision with vibrotactile, auditory, and multisensory stimuli.

The growing use of vibrotactile signaling devices makes it important to understand the perceptual limits on vibrotactile information processing. To promote that understanding, we carried out a pair of experiments on vibrotactile, auditory, and bimodal (synchronous vibrotactile and auditory) temporal acuity. On each trial, subjects experienced a set of isochronous, standard intervals (400 ms each), followed by one interval of variable duration (400 ± 1-80 ms). Intervals were demarcated by short vibrotactile, auditory, or bimodal pulses. Subjects categorized the timing of the last interval by describing the final pulse as either "early" or "late" relative to its predecessors. In Experiment 1, each trial contained three isochronous standard intervals, followed by an interval of variable length. In Experiment 2, the number of isochronous standard intervals per trial varied, from one to four. Psychometric modeling revealed that vibrotactile stimulation produced poorer temporal discrimination than either auditory or bimodal stimulation. Moreover, auditory signals dominated bimodal sensitivity, and inter-individual differences in temporal discriminability were reduced with bimodal stimulation. Additionally, varying the number of isochronous intervals in a trial failed to improve temporal sensitivity in either modality, suggesting that memory played a key role in judgments of interval duration.

Cultural differences in performance on Eriksen's flanker task.

Eriksen's zoom model of attention implies a trade-off between the breadth and resolution of representations of information. Following this perspective, we used Eriksen's flanker task to investigate culture's influence on attentional allocation and attentional resolution. In Experiment 1, the spatial distance of the flankers was varied to test whether people from Eastern cultures (here, Turks) experienced more interference than people from Western cultures (here, Americans) when flankers were further from the target. In Experiment 2, the contrast of the flankers was varied. The pattern of results shows that congruency of the flankers (Experiment 1) as well as the degree of contrast of the flankers compared with the target (Experiment 2) interact with participants' cultural background to differentially influence accuracy or reaction times. In addition, we used evidence accumulation modeling to jointly consider measures of speed and accuracy. Results indicate that to make decisions in the Eriksen flanker task, Turks both accumulate evidence faster and require more evidence than Americans do. These cultural differences in visual attention and decision-making have implications for a wide variety of cognitive processes.

Neural correlates of sequence encoding in visuomotor learning.

To examine the neural basis of sequence learning, a fundamental but poorly understood human ability, we recorded event-related potentials (ERPs) while subjects viewed and memorized randomly directed sequences of motions for later imitation. Previously, we found that the amplitude of ERPs elicited by successive motion segments decreased as a function of each segment's serial position. This happened when subjects were required to remember the sequence, but not when they were performing a perceptual task. Here, to study the functional significance of this amplitude gradient in sequence learning, we presented each sequence several times in succession and examined changes in ERP amplitude as subjects learned the sequence through repeated observation and imitation. Behaviorally, with each repetition subjects grew more accurate in reproducing what they had seen. At the same time, ERPs grew smaller with each successive presentation, replicating and extending previous demonstrations of repetition suppression. Importantly, a comparison of ERPs to segments occupying different serial positions within a sequence revealed a decreasing amplitude gradient that grew steeper with sequence repetition. This sharpening of the amplitude gradient may reflect an explicit encoding process that relies on a magnitude code for serial order.

Auditory short-term memory behaves like visual short-term memory.

Are the information processing steps that support short-term sensory memory common to all the senses? Systematic, psychophysical comparison requires identical experimental paradigms and comparable stimuli, which can be challenging to obtain across modalities. Participants performed a recognition memory task with auditory and visual stimuli that were comparable in complexity and in their neural representations at early stages of cortical processing. The visual stimuli were static and moving Gaussian-windowed, oriented, sinusoidal gratings (Gabor patches); the auditory stimuli were broadband sounds whose frequency content varied sinusoidally over time (moving ripples). Parallel effects on recognition memory were seen for number of items to be remembered, retention interval, and serial position. Further, regardless of modality, predicting an item's recognizability requires taking account of (1) the probe's similarity to the remembered list items (summed similarity), and (2) the similarity between the items in memory (inter-item homogeneity). A model incorporating both these factors gives a good fit to recognition memory data for auditory as well as visual stimuli. In addition, we present the first demonstration of the orthogonality of summed similarity and inter-item homogeneity effects. These data imply that auditory and visual representations undergo very similar transformations while they are encoded and retrieved from memory.

Sex differences in the acquisition of complex skilled movements.

The praxis system comprises a network of brain regions dedicated to complex skilled movements. Following suggestions of a female advantage on learned movements (Chipman and Hampson in Neuropsychologia 44(12):2315-2329, 2006), we investigated how males and females acquire skilled movements. Subjects viewed and imitated sequences of hand movements, which were repeated ten times. Subjects' imitations were captured by a data glove, and the temporal and spatial characteristics of the imitations were compared to the model sequence. We propose an account of the computations required for imitating hand movement sequences and define the errors that may arise from failures at these computations. Our results demonstrate a female advantage in the acquisition of hand movement sequences and show that this sex difference is accounted for by a female advantage in planning, rather than an advantage in execution or online control. Further, the female advantage is specific to the production of items within a sequence and does not affect the proper ordering of items. Our findings have important implications for sex-sensitive instructional strategies, as well as for understanding the human praxis system.