Visual and audiovisual effects of isochronous timing on visual perception and brain activity.

Understanding how the brain extracts and combines temporal structure (rhythm) information from events presented to different senses remains unresolved. Many neuroimaging beat perception studies have focused on the auditory domain and show the presence of a highly regular beat (isochrony) in "auditory" stimulus streams enhances neural responses in a distributed brain network and affects perceptual performance. Here, we acquired functional magnetic resonance imaging (fMRI) measurements of brain activity while healthy human participants performed a visual task on isochronous versus randomly timed "visual" streams, with or without concurrent task-irrelevant sounds. We found that visual detection of higher intensity oddball targets was better for isochronous than randomly timed streams, extending previous auditory findings to vision. The impact of isochrony on visual target sensitivity correlated positively with fMRI signal changes not only in visual cortex but also in auditory sensory cortex during audiovisual presentations. Visual isochrony activated a similar timing-related brain network to that previously found primarily in auditory beat perception work. Finally, activity in multisensory left posterior superior temporal sulcus increased specifically during concurrent isochronous audiovisual presentations. These results indicate that regular isochronous timing can modulate visual processing and this can also involve multisensory audiovisual brain mechanisms.

Audiovisual synchrony enhances BOLD responses in a brain network including multisensory STS while also enhancing target-detection performance for both modalities.

The brain seeks to combine related inputs from different senses (e.g., hearing and vision), via multisensory integration. Temporal information can indicate whether stimuli in different senses are related or not. A recent human fMRI study (Noesselt et al. [2007]: J Neurosci 27:11431-11441) used auditory and visual trains of beeps and flashes with erratic timing, manipulating whether auditory and visual trains were synchronous or unrelated in temporal pattern. A region of superior temporal sulcus (STS) showed higher BOLD signal for the synchronous condition. But this could not be related to performance, and it remained unclear if the erratic, unpredictable nature of the stimulus trains was important. Here we compared synchronous audiovisual trains to asynchronous trains, while using a behavioral task requiring detection of higher-intensity target events in either modality. We further varied whether the stimulus trains had predictable temporal pattern or not. Synchrony (versus lag) between auditory and visual trains enhanced behavioral sensitivity (d') to intensity targets in either modality, regardless of predictable versus unpredictable patterning. The analogous contrast in fMRI revealed BOLD increases in several brain areas, including the left STS region reported by Noesselt et al. [2007: J Neurosci 27:11431-11441]. The synchrony effect on BOLD here correlated with the subject-by-subject impact on performance. Predictability of temporal pattern did not affect target detection performance or STS activity, but did lead to an interaction with audiovisual synchrony for BOLD in inferior parietal cortex.

From threat to fear: the neural organization of defensive fear systems in humans.

Postencounter and circa-strike defensive contexts represent two adaptive responses to potential and imminent danger. In the context of a predator, the postencounter reflects the initial detection of the potential threat, whereas the circa-strike is associated with direct predatory attack. We used functional magnetic resonance imaging to investigate the neural organization of anticipation and avoidance of artificial predators with high or low probability of capturing the subject across analogous postencounter and circa-strike contexts of threat. Consistent with defense systems models, postencounter threat elicited activity in forebrain areas, including subgenual anterior cingulate cortex (sgACC), hippocampus, and amygdala. Conversely, active avoidance during circa-strike threat increased activity in mid-dorsal ACC and midbrain areas. During the circa-strike condition, subjects showed increased coupling between the midbrain and mid-dorsal ACC and decreased coupling with the sgACC, amygdala, and hippocampus. Greater activity was observed in the right pregenual ACC for high compared with low probability of capture during circa-strike threat. This region showed decreased coupling with the amygdala, insula, and ventromedial prefrontal cortex. Finally, we found that locomotor errors correlated with subjective reports of panic for the high compared with low probability of capture during the circa-strike threat, and these panic-related locomotor errors were correlated with midbrain activity. These findings support models suggesting that higher forebrain areas are involved in early-threat responses, including the assignment and control of fear, whereas imminent danger results in fast, likely "hard-wired," defensive reactions mediated by the midbrain.

Choking on the money: reward-based performance decrements are associated with midbrain activity.

A pernicious paradox in human motivation is the occasional reduced performance associated with tasks and situations that involve larger-than-average rewards. Three broad explanations that might account for such performance decrements are attentional competition (distraction theories), inhibition by conscious processes (explicit-monitoring theories), and excessive drive and arousal (overmotivation theories). Here, we report incentive-dependent performance decrements in humans in a reward-pursuit task; subjects were less successful in capturing a more valuable reward in a computerized maze. Concurrent functional magnetic resonance imaging revealed that increased activity in ventral midbrain, a brain area associated with incentive motivation and basic reward responding, correlated with both reduced number of captures and increased number of near-misses associated with imminent high rewards. These data cast light on the neurobiological basis of choking under pressure and are consistent with overmotivation accounts.

When Pinocchio's nose does not grow: belief regarding lie-detectability modulates production of deception.

Does the brain activity underlying the production of deception differ depending on whether or not one believes their deception can be detected? To address this question, we had participants commit a mock theft in a laboratory setting, and then interrogated them while they underwent functional MRI (fMRI) scanning. Crucially, during some parts of the interrogation participants believed a lie-detector was activated, whereas in other parts they were told it was switched-off. We were thus able to examine the neural activity associated with the contrast between producing true vs. false claims, as well as the independent contrast between believing that deception could and could not be detected. We found increased activation in the right amygdala and inferior frontal gyrus (IFG), as well as the left posterior cingulate cortex (PCC), during the production of false (compared to true) claims. Importantly, there was a significant interaction between the effects of deception and belief in the left temporal pole and right hippocampus/parahippocampal gyrus, where activity increased during the production of deception when participants believed their false claims could be detected, but not when they believed the lie-detector was switched-off. As these regions are associated with binding socially complex perceptual input and memory retrieval, we conclude that producing deceptive behavior in a context in which one believes this deception can be detected is associated with a cognitively taxing effort to reconcile contradictions between one's actions and recollections.

An fMRI study of joint action-varying levels of cooperation correlates with activity in control networks.

As social agents, humans continually interact with the people around them. Here, motor cooperation was investigated using a paradigm in which pairs of participants, one being scanned with fMRI, jointly controlled a visually presented object with joystick movements. The object oscillated dynamically along two dimensions, color and width of gratings, corresponding to the two cardinal directions of joystick movements. While the overall control of each participant on the object was kept constant, the amount of cooperation along the two dimensions varied along four levels, from no (each participant controlled one dimension exclusively) to full (each participant controlled half of each dimension) cooperation. Increasing cooperation correlated with BOLD signal in the left parietal operculum and anterior cingulate cortex (ACC), while decreasing cooperation correlated with activity in the right inferior frontal and superior temporal gyri, the intraparietal sulci and inferior temporal gyri bilaterally, and the dorsomedial prefrontal cortex. As joint performance improved with the level of cooperation, we assessed the brain responses correlating with behavior, and found that activity in most of the areas associated with levels of cooperation also correlated with the joint performance. The only brain area found exclusively in the negative correlation with cooperation was in the dorso medial frontal cortex, involved in monitoring action outcome. Given the cluster location and condition-related signal change, we propose that this region monitored actions to extract the level of cooperation in order to optimize the joint response. Our results, therefore, indicate that, in the current experimental paradigm involving joint control of a visually presented object with joystick movements, the level of cooperation affected brain networks involved in action control, but not mentalizing.

What if I Get Busted? Deception, Choice, and Decision-Making in Social Interaction.

Deception is an essentially social act, yet little is known about how social consequences affect the decision to deceive. In this study, participants played a computerized game of deception without constraints on whether or when to attempt to deceive their opponent. Participants were questioned by an opponent outside the scanner about their knowledge of the content of a display. Importantly, questions were posed so that, in some conditions, it was possible to be deceptive, while in other conditions it was not. To simulate a realistic interaction, participants could be confronted about their claims by the opponent. This design, therefore, creates a context in which a deceptive participant runs the risk of being punished if their deception is detected. Our results show that participants were slower to give honest than to give deceptive responses when they knew more about the display and could use this knowledge for their own benefit. The condition in which confrontation was not possible was associated with increased activity in subgenual anterior cingulate cortex. The processing of a question which allows a deceptive response was associated with activation in right caudate and inferior frontal gyrus. Our findings suggest the decision to deceive is affected by the potential risk of social confrontation rather than the claim itself.

Neural correlates of observing pretend play in which one object is represented as another.

Observers were scanned while they watched a video of an actor using an object. Three conditions were contrasted in which the same object was used: (i) normally (e.g. using a tennis racket to hit a ball), (ii) in an unusual way (e.g. using a tennis racket to strain spaghetti), (iii) in a pretend play (e.g. playing a tennis racket like a banjo). Observing real and unusual uses of objects activated areas previously seen in studies of tool use including areas associated with a mirror system for action. Observing pretend play activated additional areas previously associated with theory of mind tasks and listening to narrative, including medial prefrontal cortex, posterior superior temporal sulcus and temporal poles. After presentation of each video, observers were asked to name the object as used in the preceding action video (e.g. racket, sieve or banjo). Naming the pretend object elicited activity in medial prefrontal cortex. These results are consistent with proposals that pretend play is a form of communicative narrative, associated with the ability to mentalize. However, this leaves open the question as to whether pretence or mentalizing is the more basic process.

Relationship between activity in human primary motor cortex during action observation and the mirror neuron system.

The attenuation of the beta cortical oscillations during action observation has been interpreted as evidence of a mirror neuron system (MNS) in humans. Here we investigated the modulation of beta cortical oscillations with the viewpoint of an observed action. We asked subjects to observe videos of an actor making a variety of arm movements. We show that when subjects were observing arm movements there was a significant modulation of beta oscillations overlying left and right sensorimotor cortices. This pattern of attenuation was driven by the side of the screen on which the observed movement occurred and not by the hand that was observed moving. These results are discussed in terms of the firing patterns of mirror neurons in F5 which have been reported to have similar properties.

When fear is near: threat imminence elicits prefrontal-periaqueductal gray shifts in humans.

Humans, like other animals, alter their behavior depending on whether a threat is close or distant. We investigated spatial imminence of threat by developing an active avoidance paradigm in which volunteers were pursued through a maze by a virtual predator endowed with an ability to chase, capture, and inflict pain. Using functional magnetic resonance imaging, we found that as the virtual predator grew closer, brain activity shifted from the ventromedial prefrontal cortex to the periaqueductal gray. This shift showed maximal expression when a high degree of pain was anticipated. Moreover, imminence-driven periaqueductal gray activity correlated with increased subjective degree of dread and decreased confidence of escape. Our findings cast light on the neural dynamics of threat anticipation and have implications for the neurobiology of human anxiety-related disorders.

Modulation of the mirror system by social relevance.

When we observe the actions of others, certain areas of the brain are activated in a similar manner as to when we perform the same actions ourselves. This 'mirror system' includes areas in the ventral premotor cortex and the inferior parietal lobule. Experimental studies suggest that action observation automatically elicits activity in the observer, which precisely mirrors the activity observed. In this case we would expect this activity to be independent of observer's viewpoint. Here we use whole-head magnetoencephalography (MEG) to record cortical activity of human subjects whilst they watched a series of videos of an actor making a movement recorded from different viewpoints. We show that one cortical response to action observation (oscillatory activity in the 7-12 Hz frequency range) is modulated by the relationship between the observer and the actor. We suggest that this modulation reflects a mechanism that filters information into the 'mirror system', allowing only socially relevant information to pass.