Macaque claustrum, pulvinar and putative dorsolateral amygdala support the cross-modal association of social audio-visual stimuli based on meaning.

Social communication draws on several cognitive functions such as perception, emotion recognition and attention. The association of audio-visual information is essential to the processing of species-specific communication signals. In this study, we use functional magnetic resonance imaging in order to identify the subcortical areas involved in the cross-modal association of visual and auditory information based on their common social meaning. We identified three subcortical regions involved in audio-visual processing of species-specific communicative signals: the dorsolateral amygdala, the claustrum and the pulvinar. These regions responded to visual, auditory congruent and audio-visual stimulations. However, none of them was significantly activated when the auditory stimuli were semantically incongruent with the visual context, thus showing an influence of visual context on auditory processing. For example, positive vocalization (coos) activated the three subcortical regions when presented in the context of positive facial expression (lipsmacks) but not when presented in the context of negative facial expression (aggressive faces). In addition, the medial pulvinar and the amygdala presented multisensory integration such that audiovisual stimuli resulted in activations that were significantly higher than those observed for the highest unimodal response. Last, the pulvinar responded in a task-dependent manner, along a specific spatial sensory gradient. We propose that the dorsolateral amygdala, the claustrum and the pulvinar belong to a multisensory network that modulates the perception of visual socioemotional information and vocalizations as a function of the relevance of the stimuli in the social context. SIGNIFICANCE STATEMENT: Understanding and correctly associating socioemotional information across sensory modalities, such that happy faces predict laughter and escape scenes predict screams, is essential when living in complex social groups. With the use of functional magnetic imaging in the awake macaque, we identify three subcortical structures-dorsolateral amygdala, claustrum and pulvinar-that only respond to auditory information that matches the ongoing visual socioemotional context, such as hearing positively valenced coo calls and seeing positively valenced mutual grooming monkeys. We additionally describe task-dependent activations in the pulvinar, organizing along a specific spatial sensory gradient, supporting its role as a network regulator.

Non-uniform phase sensitivity in spatial frequency maps of the human visual cortex.

KEY POINTS: Just as a portrait painting can come from a collection of coarse and fine details, natural vision can be decomposed into coarse and fine components. Previous studies have shown that the early visual areas in the brain represent these components in a map-like fashion. Other studies have shown that these same visual areas can be sensitive to how coarse and fine features line up in space. We found that the brain actually jointly represents both the scale of the feature (fine, medium, or coarse) and the alignment of these features in space. The results suggest that the visual cortex has an optimized representation particularly for the alignment of fine details, which are crucial in understanding the visual scene. ABSTRACT: Complex natural scenes can be decomposed into their oriented spatial frequency (SF) and phase relationships, both of which are represented locally at the earliest stages of cortical visual processing. The SF preference map in the human cortex, obtained using synthetic stimuli, is orderly and correlates strongly with eccentricity. In addition, early visual areas show sensitivity to the phase information that describes the relationship between SFs and thereby dictates the structure of the image. Taken together, two possibilities arise for the joint representation of SF and phase: either the entirety of the cortical SF map is uniformly sensitive to phase, or a particular set of SFs is selectively phase sensitive - for example, greater phase sensitivity for higher SFs that define fine-scale edges in a complex scene. To test between these two possibilities, we constructed a novel continuous natural scene video whereby phase information was maintained in one SF band but scrambled elsewhere. By shifting the central frequency of the phase-aligned band in time, we mapped the phase-sensitive SF preference of the visual cortex. Using functional magnetic resonance imaging, we found that phase sensitivity in early visual areas is biased toward higher SFs. Compared to a SF map of the same scene obtained using linear-filtered stimuli, a much larger patch of areas V1 and V2 is sensitive to the phase alignment of higher SFs. The results of early areas cannot be explained by attention. Our results suggest non-uniform sensitivity to phase alignment in population-level SF representations, with phase alignment being particularly important for fine-scale edge representations of natural scenes.

High-resolution retinotopic maps estimated with magnetoencephalography.

Magnetoencephalography (MEG) is used in clinical and fundamental studies of brain functions, primarily for the excellent temporal resolution it provides. The spatial resolution is often assumed to be poor, because of the ill-posed nature of MEG source modeling. However, the question of spatial resolution in MEG has seldom been studied in quantitative detail. Here we use the well-known retinotopic organization of the primary visual cortex (V1) as a benchmark for estimating the spatial resolution of MEG source imaging. Using a standard visual stimulation paradigm in human subjects, we find that individual MEG sources exhibit well-delineated visual receptive fields that collectively follow the known mapping of the retinal surface onto the cortex. Based on the size of these receptive fields and the variability of the signal, we are able to resolve MEG signals separated by approximately 7 mm in smooth regions of cortex and less than 1 mm for signals near curved gyri. The maximum resolution is thus comparable to that of the spacing of hypercolumns in human visual cortex. Overall, our results suggest that the spatial resolution of MEG can approach or in some cases exceed that of fMRI.

Is the Cortical Deficit in Amblyopia Due to Reduced Cortical Magnification, Loss of Neural Resolution, or Neural Disorganization?

The neural basis of amblyopia is a matter of debate. The following possibilities have been suggested: loss of foveal cells, reduced cortical magnification, loss of spatial resolution of foveal cells, and topographical disarray in the cellular map. To resolve this we undertook a population receptive field (pRF) functional magnetic resonance imaging analysis in the central field in humans with moderate-to-severe amblyopia. We measured the relationship between averaged pRF size and retinal eccentricity in retinotopic visual areas. Results showed that cortical magnification is normal in the foveal field of strabismic amblyopes. However, the pRF sizes are enlarged for the amblyopic eye. We speculate that the pRF enlargement reflects loss of cellular resolution or an increased cellular positional disarray within the representation of the amblyopic eye. SIGNIFICANCE STATEMENT: The neural basis of amblyopia, a visual deficit affecting 3% of the human population, remains a matter of debate. We undertook the first population receptive field functional magnetic resonance imaging analysis in participants with amblyopia and compared the projections from the amblyopic and fellow normal eye in the visual cortex. The projection from the amblyopic eye was found to have a normal cortical magnification factor, enlarged population receptive field sizes, and topographic disorganization in all early visual areas. This is consistent with an explanation of amblyopia as an immature system with a normal complement of cells whose spatial resolution is reduced and whose topographical map is disordered. This bears upon a number of competing theories for the psychophysical defect and affects future treatment therapies.

Short-term monocular deprivation strengthens the patched eye's contribution to binocular combination.

Binocularity is a fundamental property of primate vision. Ocular dominance describes the perceptual weight given to the inputs from the two eyes in their binocular combination. There is a distribution of sensory dominance within the normal binocular population with most subjects having balanced inputs while some are dominated by the left eye and some by the right eye. Using short-term monocular deprivation, the sensory dominance can be modulated as, under these conditions, the patched eye's contribution is strengthened. We address two questions: Is this strengthening a general effect such that it is seen for different types of sensory processing? And is the strengthening specific to pattern deprivation, or does it also occur for light deprivation? Our results show that the strengthening effect is a general finding involving a number of sensory functions, and it occurs as a result of both pattern and light deprivation.

Neural mechanisms of understanding rational actions: middle temporal gyrus activation by contextual violation.

Performing goal-directed actions toward an object in accordance with contextual constraints, such as the presence or absence of an obstacle, has been widely used as a paradigm for assessing the capacity of infants or nonhuman primates to evaluate the rationality of others' actions. Here, we have used this paradigm in a functional magnetic resonance imaging experiment to visualize the cortical regions involved in the assessment of action rationality while controlling for visual differences in the displays and directly correlating magnetic resonance activity with rationality ratings. Bilateral middle temporal gyrus (MTG) regions, anterior to extrastriate body area and the human middle temporal complex, were involved in the visual evaluation of action rationality. These MTG regions are embedded in the superior temporal sulcus regions processing the kinematics of observed actions. Our results suggest that rationality is assessed initially by purely visual computations, combining the kinematics of the action with the physical constraints of the environmental context. The MTG region seems to be sensitive to the contingent relationship between a goal-directed biological action and its relevant environmental constraints, showing increased activity when the expected pattern of rational goal attainment is violated.

Socially meaningful visual context either enhances or inhibits vocalisation processing in the macaque brain.

Social interactions rely on the interpretation of semantic and emotional information, often from multiple sensory modalities. Nonhuman primates send and receive auditory and visual communicative signals. However, the neural mechanisms underlying the association of visual and auditory information based on their common social meaning are unknown. Using heart rate estimates and functional neuroimaging, we show that in the lateral and superior temporal sulcus of the macaque monkey, neural responses are enhanced in response to species-specific vocalisations paired with a matching visual context, or when vocalisations follow, in time, visual information, but inhibited when vocalisation are incongruent with the visual context. For example, responses to affiliative vocalisations are enhanced when paired with affiliative contexts but inhibited when paired with aggressive or escape contexts. Overall, we propose that the identified neural network represents social meaning irrespective of sensory modality.

Object-based neglect varies with egocentric position.

Different reference frames have been identified to influence neglect behavior. In particular, neglect has been demonstrated to be related to the contralesional side of the subject's body (egocentric reference frames) as well as to the contralesional side of individual objects irrespective of their position to the patient (object-based reference frame). There has been discussion whether this distinction separates neglect into body- and object-based forms. The present experiment aimed to prove possible interactions between object-based and egocentric aspects in spatial neglect. Neglect patients' eye and head movements were recorded while they explored objects at five egocentric positions along the horizontal dimension of space. The patients showed both egocentric as well as object-based behavior. Most interestingly, data analysis revealed that object-based neglect varied with egocentric position. Although the neglect of the objects' left side was strong at contralesional egocentric positions, it ameliorated at more ipsilesional egocentric positions of the objects. The patients showed steep, ramp-shaped patterns of exploration for objects located on the far contralesional side and a broadening of these patterns as the locations of the objects shifted more to the ipsilesional side. The data fitted well with the saliency curves predicted by a model of space representation, which suggests that visual input is represented in two modes simultaneously: in veridical egocentric coordinates and in within-object coordinates.

Two cortical systems for reaching in central and peripheral vision.

Parietal lesions in humans can produce a specific disruption of visually guided hand movement, termed optic ataxia. The fact that the deficit mainly occurs in peripheral vision suggests that reaching in foveal and extrafoveal vision rely on two different neural substrates. In the present study, we have directly tested this hypothesis by event-related fMRI in healthy subjects. Brain activity was measured when participants reached toward central or peripheral visual targets. Our results confirm the existence of two systems, differently modulated by the two conditions. Reaching in central vision involved a restricted network including the medial intraparietal sulcus (mIPS) and the caudal part of the dorsal premotor cortex (PMd). Reaching in peripheral vision activated in addition the parieto-occipital junction (POJ) and a more rostral part of PMd. These results show that reaching to the peripheral visual field engages a more extensive cortical network than reaching to the central visual field.

How humans reach: distinct cortical systems for central and peripheral vision.

Lesions of the posterior parietal cortex in humans can produce a specific disruption of visually guided hand movements termed optic ataxia. The fact that the deficit mainly occurs in peripheral vision suggests that reaching in foveal and extrafoveal vision relies on two different anatomical substrates. Using fMRI in healthy subjects, the authors demonstrated the existence of two systems, differently modulated by the two reaching conditions. Reaching in central vision involves a restricted network, including the medial intraparietal sulcus (mIPS) and the caudal part of the dorsal premotor cortex (PMd). Reaching in peripheral vision engages a more extensive network, including the parieto-occipital junction (POJ). Interestingly, POJ corresponds to the site of the lesion overlap that the authors recently found to be responsible for optic ataxia. These two sets of results converge to show that there is not a unique cortical network for reaching control but instead two systems engaged in reaching to targets in the central and peripheral visual field.

Latent binocular function in amblyopia.

Recently, psychophysical studies have shown that humans with amblyopia do have binocular function that is not normally revealed due to dominant suppressive interactions under normal viewing conditions. Here we use magnetoencephalography (MEG) combined with dichoptic visual stimulation to investigate the underlying binocular function in humans with amblyopia for stimuli that, because of their temporal properties, would be expected to bypass suppressive effects and to reveal any underlying binocular function. We recorded contrast response functions in visual cortical area V1 of amblyopes and normal observers using a steady state visually evoked responses (SSVER) protocol. We used stimuli that were frequency-tagged at 4Hz and 6Hz that allowed identification of the responses from each eye and were of a sufficiently high temporal frequency (>3Hz) to bypass suppression. To characterize binocular function, we compared dichoptic masking between the two eyes in normal and amblyopic participants as well as interocular phase differences in the two groups. We observed that the primary visual cortex responds less to the stimulation of the amblyopic eye compared to the fellow eye. The pattern of interaction in the amblyopic visual system however was not significantly different between the amblyopic and fellow eyes. However, the amblyopic suppressive interactions were lower than those observed in the binocular system of our normal observers. Furthermore, we identified an interocular processing delay of approximately 20ms in our amblyopic group. To conclude, when suppression is greatly reduced, such as the case with our stimulation above 3Hz, the amblyopic visual system exhibits a lack of binocular interactions.

Short-term monocular occlusion produces changes in ocular dominance by a reciprocal modulation of interocular inhibition.

Ocular dominance can be modulated by short-term monocular deprivation. This changes the contribution that each eye makes to binocular vision, an example of adult cortical neuroplasticity. Optical imaging in primates and psychophysics in humans suggest these neuroplastic changes occur in V1. Here we use brain imaging (MEG) in normal adults to better understand the nature of these neuroplastic changes. The results suggest that short-term monocular deprivation, whether it be by an opaque or translucent patch, modulates dichoptic inhibitory interactions in a reciprocal fashion; the unpatched eye is inhibited, the patched eye is released from inhibition. These observations locate the neuroplastic changes to a level of visual processing where there are interocular inhibitory interactions prior to binocular combination and help to explain why both binocular rivalry and fusional tasks reveal them.

Regional Extent of Peripheral Suppression in Amblyopia.

Purpose: Previously, we have mapped amblyopic eye suppression within the central 20° of the visual field and observed a gradient of suppression that is strongest in central vision and weakens with increasing eccentricity. In this study, using a large dichoptic display, we extend our novel suppression mapping approach further into the periphery (from 20°-60°) to assess whether suppression continues to decline with eccentricity or plateaus. Methods: Sixteen participants with amblyopia (10 with strabismus, 6 with anisometropia without strabismus; mean age: 37.9 ± 11 years) and six normal observers (mean age: 28.3 ± 5 years) took part. The visual stimulus (60° diameter), viewed from 57 cm, was composed of four concentric annuli (5° radius) with alternate contrast polarities starting from an eccentricity of 10°. Each annulus was divided into eight sectors subtending 45° of visual angle. Participants adjusted the contrast of a single sector presented to the fellow eye to match the perceived contrast of the remaining stimulus elements that were presented to the amblyopic eye. A matching contrast that was lower in the fellow eye than the amblyopic eye indicated suppression. Results: Patients with strabismus exhibited significantly stronger interocular suppression than controls across all eccentricities (P = 0.01). Patients with anisometropia did not differ from controls (P = 0.58). Suppression varied significantly with eccentricity (P = 0.005) but this effect did not differ between patient groups (P = 0.217). Conclusions: In amblyopia, suppression is present beyond the central 10° in patients with strabismus. Suppression becomes weaker at greater eccentricities and this may enable peripheral fusion that could be used by binocular treatment methods.

A common basis for visual and tactile exploration deficits in spatial neglect?

One of the fundamental characteristics of spatial neglect is an imbalanced visual search behaviour favouring stimuli on the right side of space while largely ignoring those on the left side. Opinions differ as to whether this reflects a general orientational bias caused by impaired supramodal body-centred reference systems, or a modality-specific search disorder. A prediction of the former model would be that exploratory activity is similarly impaired both in vision and in the absence of visual control. We addressed this hypothesis by comparing patients' visual and tactile search in the same workspace. Our results show that the centre of exploration activity in both modalities was substantially shifted towards the ipsilesional right side in the neglect group as compared to healthy and patient controls. This bias was more accentuated for visual search. We found a clear linear relationship between the visual and tactile search biases in the patient group with spatial neglect. Our finding suggests that the critical component guiding search behaviour in neglect, whether visually or tactually, is a general rightward orientation bias. In addition, we observed an increased repetition rate in both modalities which affected the whole workspace. This implies that the apparent spatial working memory deficit dissociates from the mechanisms inducing the orientation bias.

Restricted ocular exploration does not seem to explain simultanagnosia.

One major function of parietal cortex is to direct our attention towards salient stimuli. The present data suggest that it also plays an important role in visual gestalt perception. Patients with simultanagnosia following lesions in this area are not able to extract the meaning of a visual scene whereas being perfectly able to recognise individual objects of this scene. We tested two patients with simultanagnosia with hierarchical Navon figures combined with eye movements recordings. The patients' performance allowed us to compare directly the scan paths in trials in which the global letter shape was recognised with trials in which the global letter shape was not recognised. We did not find any obvious differences in the eye movement pattern related to the two perceptual situations. The two patients did not show a significant problem in shifting their eyes (and thus possibly also their attentional focus) to all aspects of the complex visual stimulus when attempting to bind together the different elements of spatially distributed information. The results demonstrate that restricted ocular exploration cannot be the reason for the patients' inability to recognise the global shape of stimuli. Our data rather suggest a role of parietal cortex in visual gestalt perception that is beyond its role of directing attention towards relevant objects.

A Robust and Reliable Test to Measure Stereopsis in the Clinic.

PURPOSE: The purpose of this study was to develop a convenient test of stereopsis in the clinic that is both robust and reliable and capable of providing a measure of variability necessary to make valid comparisons between measurements obtained at different occasions or under different conditions. METHODS: Stereo acuity was measured based on principles derived from the laboratory measurement of stereopsis (i.e., staircase method). Potential premeasurement compensations are described if there is a significant degree of ocular misalignment, reduced visual acuity, or aniseikonia. Forty-six adults at McGill University, 44 adults at Auckland University, and 51 adults from the University of Bradford, with an age range of 20 to 65 years old and normal or corrected-to-normal vision participated in this study. RESULTS: Stereo acuity within this normal population was widely distributed, with a significant percentage (28%) of the population with only coarse stereo (>300 arc seconds). Across subjects, the SD was approximately 25% of the mean. Measurements at two different times were strongly (r = 0.79) and significantly (P < 0.001) correlated, with little to no significant (P = 0.79) bias (0.01) between test and retest measures of stereopsis. CONCLUSIONS: The application enables measurements over the wide disparity range and not just at the finest disparities. In addition, it allows changes in stereopsis of the order of 1.9 to be statistically distinguished.

Anatomical evidence of multimodal integration in primate striate cortex.

The primary visual cortex (area 17 or V1) is not thought to receive input from nonvisual extrastriate cortical areas. However, this has yet to be shown to be the case using sensitive tracers in the part of area 17 subserving the peripheral visual field. Here we show using retrograde tracers that peripheral area 17 subserving the visual field at an eccentricity of 10-20 degrees receives projections from the core and parabelt areas of the auditory cortex as well as from the polysensory area of the temporal lobe (STP). The relative strength of these projections was calculated for each injection by computing the proportions of retrogradely labeled neurons located in the auditory and STP areas with respect to number of labeled neurons constituting the established projection from the superior temporal sulci (STS) motion complex (middle temporal area, medial superior temporal, fundus of the superior temporal area). In peripheral area V1 the projection from auditory cortex corresponds to 9.5% of that of the STS motion complex and STP to 35% of that from the STS motion complex. Compared to peripheral area 17, central and paracentral area 17 showed considerably weaker inputs from auditory cortex (0.2-0.8%) but slightly more from STP cortex (3.5-6.1%). The present results show that the connectivity of area 17 is eccentricity dependent. Direct projections from auditory and STP cortex to peripheral area 17 have important consequences for higher visual functions of area 17, including multimodal integration at early stages of the visual cortical pathway.

The iPod binocular home-based treatment for amblyopia in adults: efficacy and compliance.

BACKGROUND: Occlusion therapy for amblyopia is predicated on the idea that amblyopia is primarily a disorder of monocular vision; however, there is growing evidence that patients with amblyopia have a structurally intact binocular visual system that is rendered functionally monocular due to suppression. Furthermore, we have found that a dichoptic treatment intervention designed to directly target suppression can result in clinically significant improvement in both binocular and monocular visual function in adult patients with amblyopia. The fact that monocular improvement occurs in the absence of any fellow eye occlusion suggests that amblyopia is, in part, due to chronic suppression. Previously the treatment has been administered as a psychophysical task and more recently as a video game that can be played on video goggles or an iPod device equipped with a lenticular screen. The aim of this case-series study of 14 amblyopes (six strabismics, six anisometropes and two mixed) ages 13 to 50 years was to investigate: 1. whether the portable video game treatment is suitable for at-home use and 2. whether an anaglyphic version of the iPod-based video game, which is more convenient for at-home use, has comparable effects to the lenticular version. METHODS: The dichoptic video game treatment was conducted at home and visual functions assessed before and after treatment. RESULTS: We found that at-home use for 10 to 30 hours restored simultaneous binocular perception in 13 of 14 cases along with significant improvements in acuity (0.11 ± 0.08 logMAR) and stereopsis (0.6 ± 0.5 log units). Furthermore, the anaglyph and lenticular platforms were equally effective. In addition, the iPod devices were able to record a complete and accurate picture of treatment compliance. CONCLUSION: The home-based dichoptic iPod approach represents a viable treatment for adults with amblyopia.