Infrared neural stimulation with 7T fMRI: A rapid in vivo method for mapping cortical connections of primate amygdala.

We have previously shown that INS-fMRI is a rapid method for mapping mesoscale brain networks in the macaque monkey brain. Focal stimulation of single cortical sites led to the activation of connected cortical locations, resulting in a global connectivity map. Here, we have extended this method for mapping brainwide networks following stimulation of single subcortical sites. As a testbed, we focused on the basal nucleus of the amygdala in the macaque monkey. We describe methods to target basal nucleus locations with submillimeter precision, pulse train stimulation methods, and statistical tests for assessing non-random nature of activations. Using these methods, we report that stimulation of precisely targeted loci in the basal nucleus produced sparse and specific activations in the brain. Activations were observed in the insular and sensory association cortices as well as activations in the cingulate cortex, consistent with known anatomical connections. What is new here is that the activations were focal and, in some cases, exhibited shifting topography with millimeter shifts in stimulation site. The precision of the method enables networks mapped from different nearby sites in the basal nucleus to be distinguished. While further investigation is needed to improve the sensitivity of this method, our analyses do support the reproducibility and non-random nature of some of the activations. We suggest that INS-fMRI is a promising method for mapping large-scale cortical and subcortical networks at high spatial resolution.

The primate cortical auditory system and neural representation of conspecific vocalizations.

Over the past decade, renewed interest in the auditory system has resulted in a surge of anatomical and physiological research in the primate auditory cortex and its targets. Anatomical studies have delineated multiple areas in and around primary auditory cortex and demonstrated connectivity among these areas, as well as between these areas and the rest of the cortex, including prefrontal cortex. Physiological recordings of auditory neurons have found that species-specific vocalizations are useful in probing the selectivity and potential functions of acoustic neurons. A number of cortical regions contain neurons that are robustly responsive to vocalizations, and some auditory responsive neurons show more selectivity for vocalizations than for other complex sounds. Demonstration of selectivity for vocalizations has prompted the question of which features are encoded by higher-order auditory neurons. Results based on detailed studies of the structure of these vocalizations, as well as the tuning and information-coding properties of neurons sensitive to these vocalizations, have begun to provide answers to this question. In future studies, these and other methods may help to define the way in which cells, ensembles, and brain regions process communication sounds. Moreover, the discovery that several nonprimary auditory cortical regions may be multisensory and responsive to vocalizations with corresponding facial gestures may change the way in which we view the processing of communication information by the auditory system.

Inactivation of Primate Prefrontal Cortex Impairs Auditory and Audiovisual Working Memory.

The prefrontal cortex is associated with cognitive functions that include planning, reasoning, decision-making, working memory, and communication. Neurophysiology and neuropsychology studies have established that dorsolateral prefrontal cortex is essential in spatial working memory while the ventral frontal lobe processes language and communication signals. Single-unit recordings in nonhuman primates has shown that ventral prefrontal (VLPFC) neurons integrate face and vocal information and are active during audiovisual working memory. However, whether VLPFC is essential in remembering face and voice information is unknown. We therefore trained nonhuman primates in an audiovisual working memory paradigm using naturalistic face-vocalization movies as memoranda. We inactivated VLPFC, with reversible cortical cooling, and examined performance when faces, vocalizations or both faces and vocalization had to be remembered. We found that VLPFC inactivation impaired subjects' performance in audiovisual and auditory-alone versions of the task. In contrast, VLPFC inactivation did not disrupt visual working memory. Our studies demonstrate the importance of VLPFC in auditory and audiovisual working memory for social stimuli but suggest a different role for VLPFC in unimodal visual processing. SIGNIFICANCE STATEMENT: The ventral frontal lobe, or inferior frontal gyrus, plays an important role in audiovisual communication in the human brain. Studies with nonhuman primates have found that neurons within ventral prefrontal cortex (VLPFC) encode both faces and vocalizations and that VLPFC is active when animals need to remember these social stimuli. In the present study, we temporarily inactivated VLPFC by cooling the cortex while nonhuman primates performed a working memory task. This impaired the ability of subjects to remember a face and vocalization pair or just the vocalization alone. Our work highlights the importance of the primate VLPFC in the processing of faces and vocalizations in a manner that is similar to the inferior frontal gyrus in the human brain.

Prefrontal neuronal responses during audiovisual mnemonic processing.

During communication we combine auditory and visual information. Neurophysiological research in nonhuman primates has shown that single neurons in ventrolateral prefrontal cortex (VLPFC) exhibit multisensory responses to faces and vocalizations presented simultaneously. However, whether VLPFC is also involved in maintaining those communication stimuli in working memory or combining stored information across different modalities is unknown, although its human homolog, the inferior frontal gyrus, is known to be important in integrating verbal information from auditory and visual working memory. To address this question, we recorded from VLPFC while rhesus macaques (Macaca mulatta) performed an audiovisual working memory task. Unlike traditional match-to-sample/nonmatch-to-sample paradigms, which use unimodal memoranda, our nonmatch-to-sample task used dynamic movies consisting of both facial gestures and the accompanying vocalizations. For the nonmatch conditions, a change in the auditory component (vocalization), the visual component (face), or both components was detected. Our results show that VLPFC neurons are activated by stimulus and task factors: while some neurons simply responded to a particular face or a vocalization regardless of the task period, others exhibited activity patterns typically related to working memory such as sustained delay activity and match enhancement/suppression. In addition, we found neurons that detected the component change during the nonmatch period. Interestingly, some of these neurons were sensitive to the change of both components and therefore combined information from auditory and visual working memory. These results suggest that VLPFC is not only involved in the perceptual processing of faces and vocalizations but also in their mnemonic processing.

Responses of prefrontal multisensory neurons to mismatching faces and vocalizations.

Social communication relies on the integration of auditory and visual information, which are present in faces and vocalizations. Evidence suggests that the integration of information from multiple sources enhances perception compared with the processing of a unimodal stimulus. Our previous studies demonstrated that single neurons in the ventrolateral prefrontal cortex (VLPFC) of the rhesus monkey (Macaca mulatta) respond to and integrate conspecific vocalizations and their accompanying facial gestures. We were therefore interested in how VLPFC neurons respond differentially to matching (congruent) and mismatching (incongruent) faces and vocalizations. We recorded VLPFC neurons during the presentation of movies with congruent or incongruent species-specific facial gestures and vocalizations as well as their unimodal components. Recordings showed that while many VLPFC units are multisensory and respond to faces, vocalizations, or their combination, a subset of neurons showed a significant change in neuronal activity in response to incongruent versus congruent vocalization movies. Among these neurons, we typically observed incongruent suppression during the early stimulus period and incongruent enhancement during the late stimulus period. Incongruent-responsive VLPFC neurons were both bimodal and nonlinear multisensory, fostering their ability to respond to changes in either modality of a face-vocalization stimulus. These results demonstrate that ventral prefrontal neurons respond to changes in either modality of an audiovisual stimulus, which is important in identity processing and for the integration of multisensory communication information.

Integration of faces and vocalizations in ventral prefrontal cortex: implications for the evolution of audiovisual speech.

The integration of facial gestures and vocal signals is an essential process in human communication and relies on an interconnected circuit of brain regions, including language regions in the inferior frontal gyrus (IFG). Studies have determined that ventral prefrontal cortical regions in macaques [e.g., the ventrolateral prefrontal cortex (VLPFC)] share similar cytoarchitectonic features as cortical areas in the human IFG, suggesting structural homology. Anterograde and retrograde tracing studies show that macaque VLPFC receives afferents from the superior and inferior temporal gyrus, which provide complex auditory and visual information, respectively. Moreover, physiological studies have shown that single neurons in VLPFC integrate species-specific face and vocal stimuli. Although bimodal responses may be found across a wide region of prefrontal cortex, vocalization responsive cells, which also respond to faces, are mainly found in anterior VLPFC. This suggests that VLPFC may be specialized to process and integrate social communication information, just as the IFG is specialized to process and integrate speech and gestures in the human brain.

Faces in motion: selectivity of macaque and human face processing areas for dynamic stimuli.

Face recognition mechanisms need to extract information from static and dynamic faces. It has been hypothesized that the analysis of dynamic face attributes is performed by different face areas than the analysis of static facial attributes. To date, there is no evidence for such a division of labor in macaque monkeys. We used fMRI to determine specializations of macaque face areas for motion. Face areas in the fundus of the superior temporal sulcus responded to general object motion; face areas outside of the superior temporal sulcus fundus responded more to facial motion than general object motion. Thus, the macaque face-processing system exhibits regional specialization for facial motion. Human face areas, processing the same stimuli, exhibited specializations for facial motion as well. Yet the spatial patterns of facial motion selectivity differed across species, suggesting that facial dynamics are analyzed differently in humans and macaques.

Multisensory interactions of face and vocal information during perception and memory in ventrolateral prefrontal cortex.

The ventral frontal lobe is a critical node in the circuit that underlies communication, a multisensory process where sensory features of faces and vocalizations come together. The neural basis of face and vocal integration is a topic of great importance since the integration of multiple sensory signals is essential for the decisions that govern our social interactions. Investigations have shown that the macaque ventrolateral prefrontal cortex (VLPFC), a proposed homologue of the human inferior frontal gyrus, is involved in the processing, integration and remembering of audiovisual signals. Single neurons in VLPFC encode and integrate species-specific faces and corresponding vocalizations. During working memory, VLPFC neurons maintain face and vocal information online and exhibit selective activity for face and vocal stimuli. Population analyses indicate that identity, a critical feature of social stimuli, is encoded by VLPFC neurons and dictates the structure of dynamic population activity in the VLPFC during the perception of vocalizations and their corresponding facial expressions. These studies suggest that VLPFC may play a primary role in integrating face and vocal stimuli with contextual information, in order to support decision making during social communication. This article is part of the theme issue 'Decision and control processes in multisensory perception'.

Representation of Expression and Identity by Ventral Prefrontal Neurons.

Evidence has suggested that the ventrolateral prefrontal cortex (VLPFC) processes social stimuli, including faces and vocalizations, which are essential for communication. Features embedded within audiovisual stimuli, including emotional expression and caller identity, provide abundant information about an individual's intention, emotional state, motivation, and social status, which are important to encode in a social exchange. However, it is unknown to what extent the VLPFC encodes such features. To investigate the role of VLPFC during social communication, we recorded single-unit activity while rhesus macaques (Macaca mulatta) performed a nonmatch-to-sample task using species-specific face-vocalization stimuli that differed in emotional expression or caller identity. 75% of recorded cells were task-related and of these >70% were responsive during the nonmatch period. A larger proportion of nonmatch cells encoded the stimulus rather than the context of the trial type. A subset of responsive neurons were most commonly modulated by the identity of the nonmatch stimulus and less by the emotional expression, or both features within the face-vocalization stimuli presented during the nonmatch period. Neurons encoding identity were found in VLPFC across a broader region than expression related cells which were confined to only the anterolateral portion of the recording chamber in VLPFC. These findings suggest that, within a working memory paradigm, VLPFC processes features of face and vocal stimuli, such as emotional expression and identity, in addition to task and contextual information. Thus, stimulus and contextual information may be integrated by VLPFC during social communication.

A View from the Top: Prefrontal Control of Object Recognition.

In this issue of Neuron, Kar and DiCarlo (2021) demonstrate that feedback from ventrolateral prefrontal cortex (VLPFC) to inferotemporal cortex (IT) is required for object recognition. They show that inactivating VLPFC selectively degrades object recognition performance and population encoding of object identity in IT.

Audiovisual integration in macaque face patch neurons.

Primate social communication depends on the perceptual integration of visual and auditory cues, reflected in the multimodal mixing of sensory signals in certain cortical areas. The macaque cortical face patch network, identified through visual, face-selective responses measured with fMRI, is assumed to contribute to visual social interactions. However, whether face patch neurons are also influenced by acoustic information, such as the auditory component of a natural vocalization, remains unknown. Here, we recorded single-unit activity in the anterior fundus (AF) face patch, in the superior temporal sulcus, and anterior medial (AM) face patch, on the undersurface of the temporal lobe, in macaques presented with audiovisual, visual-only, and auditory-only renditions of natural movies of macaques vocalizing. The results revealed that 76% of neurons in face patch AF were significantly influenced by the auditory component of the movie, most often through enhancement of visual responses but sometimes in response to the auditory stimulus alone. By contrast, few neurons in face patch AM exhibited significant auditory responses or modulation. Control experiments in AF used an animated macaque avatar to demonstrate, first, that the structural elements of the face were often essential for audiovisual modulation and, second, that the temporal modulation of the acoustic stimulus was more important than its frequency spectrum. Together, these results identify a striking contrast between two face patches and specifically identify AF as playing a potential role in the integration of audiovisual cues during natural modes of social communication.

An auditory domain in primate prefrontal cortex.

Although neuroimaging studies confirm the frontal lobe's involvement in language processes and auditory working memory, the cellular and network basis of these functions is unclear. Physiological studies of the frontal lobe in non-human primates have focused on visual working memory and auditory spatial processing in dorsolateral prefrontal cortex (PFC), although the candidate PFC areas for non-spatial acoustic processing lie in the ventrolateral PFC (areas 12 and 45), which receives afferents from physiologically and anatomically defined auditory cortex. We recorded neuronal responses from ventrolateral PFC to auditory cues in awake monkeys under controlled conditions and report that the macaque ventrolateral PFC contains an auditory responsive domain in which neurons show responses to complex sounds, including animal and human vocalizations.

Probabilistic encoding of vocalizations in macaque ventral lateral prefrontal cortex.

We examined strategies for classifying macaque vocalizations into their corresponding categories, as well as whether or not there was evidence that prefrontal auditory neurons were related to this process. We found that static estimates of the spectral and temporal contrasts of the calls were not effective features for discriminating among the call classes. A hidden Markov model (HMM), however, was more effective at discriminating among the call classes, reaching a performance of almost 75% correct. Finally, we found that the responses of prefrontal auditory neurons could be predicted more effectively as linear functions of the probabilistic output of the HMM than as linear functions of the spectral features of the calls. This provides evidence that, for call recognition, the macaque auditory system likely performs dynamic processing of vocalizations, and that prefrontal auditory neurons carry a signal related to the output of this processing.

Integration of auditory and visual communication information in the primate ventrolateral prefrontal cortex.

The integration of auditory and visual stimuli is crucial for recognizing objects, communicating effectively, and navigating through our complex world. Although the frontal lobes are involved in memory, communication, and language, there has been no evidence that the integration of communication information occurs at the single-cell level in the frontal lobes. Here, we show that neurons in the macaque ventrolateral prefrontal cortex (VLPFC) integrate audiovisual communication stimuli. The multisensory interactions included both enhancement and suppression of a predominantly auditory or a predominantly visual response, although multisensory suppression was the more common mode of response. The multisensory neurons were distributed across the VLPFC and within previously identified unimodal auditory and visual regions (O'Scalaidhe et al., 1997; Romanski and Goldman-Rakic, 2002). Thus, our study demonstrates, for the first time, that single prefrontal neurons integrate communication information from the auditory and visual domains, suggesting that these neurons are an important node in the cortical network responsible for communication.

Auditory connections and functions of prefrontal cortex.

The functional auditory system extends from the ears to the frontal lobes with successively more complex functions occurring as one ascends the hierarchy of the nervous system. Several areas of the frontal lobe receive afferents from both early and late auditory processing regions within the temporal lobe. Afferents from the early part of the cortical auditory system, the auditory belt cortex, which are presumed to carry information regarding auditory features of sounds, project to only a few prefrontal regions and are most dense in the ventrolateral prefrontal cortex (VLPFC). In contrast, projections from the parabelt and the rostral superior temporal gyrus (STG) most likely convey more complex information and target a larger, widespread region of the prefrontal cortex. Neuronal responses reflect these anatomical projections as some prefrontal neurons exhibit responses to features in acoustic stimuli, while other neurons display task-related responses. For example, recording studies in non-human primates indicate that VLPFC is responsive to complex sounds including vocalizations and that VLPFC neurons in area 12/47 respond to sounds with similar acoustic morphology. In contrast, neuronal responses during auditory working memory involve a wider region of the prefrontal cortex. In humans, the frontal lobe is involved in auditory detection, discrimination, and working memory. Past research suggests that dorsal and ventral subregions of the prefrontal cortex process different types of information with dorsal cortex processing spatial/visual information and ventral cortex processing non-spatial/auditory information. While this is apparent in the non-human primate and in some neuroimaging studies, most research in humans indicates that specific task conditions, stimuli or previous experience may bias the recruitment of specific prefrontal regions, suggesting a more flexible role for the frontal lobe during auditory cognition.

Coding of vocalizations by single neurons in ventrolateral prefrontal cortex.

Neuronal activity in single prefrontal neurons has been correlated with behavioral responses, rules, task variables and stimulus features. In the non-human primate, neurons recorded in ventrolateral prefrontal cortex (VLPFC) have been found to respond to species-specific vocalizations. Previous studies have found multisensory neurons which respond to simultaneously presented faces and vocalizations in this region. Behavioral data suggests that face and vocal information are inextricably linked in animals and humans and therefore may also be tightly linked in the coding of communication calls in prefrontal neurons. In this study we therefore examined the role of VLPFC in encoding vocalization call type information. Specifically, we examined previously recorded single unit responses from the VLPFC in awake, behaving rhesus macaques in response to 3 types of species-specific vocalizations made by 3 individual callers. Analysis of responses by vocalization call type and caller identity showed that ∼19% of cells had a main effect of call type with fewer cells encoding caller. Classification performance of VLPFC neurons was ∼42% averaged across the population. When assessed at discrete time bins, classification performance reached 70 percent for coos in the first 300 ms and remained above chance for the duration of the response period, though performance was lower for other call types. In light of the sub-optimal classification performance of the majority of VLPFC neurons when only vocal information is present, and the recent evidence that most VLPFC neurons are multisensory, the potential enhancement of classification with the addition of accompanying face information is discussed and additional studies recommended. Behavioral and neuronal evidence has shown a considerable benefit in recognition and memory performance when faces and voices are presented simultaneously. In the natural environment both facial and vocalization information is present simultaneously and neural systems no doubt evolved to integrate multisensory stimuli during recognition. This article is part of a Special Issue entitled "Communication Sounds and the Brain: New Directions and Perspectives".

Representation and integration of auditory and visual stimuli in the primate ventral lateral prefrontal cortex.

Through the influence of Goldman-Rakic, much research has been focused on the role of the dorsolateral prefrontal cortex in spatial working memory, decision making, and saccade generation, whereas functions of other parts of the frontal lobe including the ventrolateral prefrontal cortex (VLPFC) are less clear. Previous studies in non-human primates have shown that some VLPFC cells are selectively responsive to faces. Recent findings indicate that adjacent to the region where face- and object-selective cells have been recorded are neurons which respond to complex sounds including human and monkey vocalizations. Furthermore, when neurons in this same region are tested with combined face and voice communication stimuli, it is apparent that some cells in VLPFC are multisensory and respond to audiovisual stimuli. The determination that ventral prefrontal neurons are multisensory and responsive to auditory and visual communication stimuli may help to establish an animal model to assist in the investigation of the circuit and cellular basis of human communication. This will also aid in the understanding of general frontal lobe function and the processes that go awry in disorders including autism and schizophrenia, where disturbances in prefrontal function have been noted.

Neural representation of vocalizations in the primate ventrolateral prefrontal cortex.

In this study, we examined the role of the ventrolateral prefrontal cortex in encoding communication stimuli. Specifically, we recorded single-unit responses from the ventrolateral prefrontal cortext (vlPFC) in awake behaving rhesus macaques in response to species-specific vocalizations. We determined the selectivity of vlPFC cells for 10 types of rhesus vocalizations and also asked what types of vocalizations cluster together in the neuronal response. The data from the present study demonstrate that vlPFC auditory neurons respond to a variety of species-specific vocalizations from a previously characterized library. Most vlPFC neurons responded to two to five vocalizations, while a small percentage of cells responded either selectively to a particular vocalization type or nonselectively to most auditory stimuli tested. Use of information theoretic approaches to examine vocalization tuning indicates that on average, vlPFC neurons encode information about one or two vocalizations. Further analysis of the types of vocalizations that vlPFC cells typically respond to using hierarchical cluster analysis suggests that the responses of vlPFC cells to multiple vocalizations is not based strictly on the call's function or meaning but may be due to other features including acoustic morphology. These data are consistent with a role for the primate vlPFC in assessing distinctive acoustic features.

Domain specificity in the primate prefrontal cortex.

Experimental studies in nonhuman primates and functional imaging studies in humans have underlined the critical role played by the prefrontal cortex (PFC) in working memory. However, the precise organization of the frontal lobes with respect to the different types of information operated upon is a point of controversy, and several models of functional organizations have been proposed. One model, developed by Goldman-Rakic and colleagues, postulates a modular organization of working memory based on the type of information processing (the domain specificity hypothesis). Evidence to date has focused on the encoding of the locations of visual objects by the dorsolateral PFC, whereas the ventrolateral PFC is suggested to be involved in processing the features and identity of objects. In this model, domain should refer to any sensory modality that registers information relevant to that domain--for example, there would be visual and auditory input to a spatial information processing region and a feature analysis system. In support of this model, recent studies have described pathways from the posterior and anterior auditory association cortex that target dorsolateral spatial-processing regions and ventrolateral object-processing regions, respectively. In addition, physiological recordings from the ventrolateral PFC indicate that some cells in this region are responsive to the features of complex sounds. Finally, recordings in adjacent ventrolateral prefrontal regions have shown that the features of somatosensory stimuli can be discriminated and encoded by ventrolateral prefrontal neurons. These discoveries argue that two domains, differing with respect to the type of information being processed, and not with respect to the sensory modality of the information, are specifically localized to discrete regions of the PFC and embody the domain specificity hypothesis, first proposed by Patricia Goldman-Rakic.

Principal and independent components of macaque vocalizations: constructing stimuli to probe high-level sensory processing.

Neurons in high-level sensory cortical areas respond to complex features in sensory stimuli. Feature elimination is a useful technique for studying these responses. In this approach, a complex stimulus, which evokes a neuronal response, is simplified, and if the cell responds to the reduced stimulus, it is considered selective for the remaining features. We have developed a feature-elimination technique that uses either the principal or the independent components of a stimulus to define a subset of features, to which a neuron might be sensitive. The original stimulus can be filtered using these components, resulting in a stimulus that retains only a fraction of the features present in the original. We demonstrate the use of this technique on macaque vocalizations, an important class of stimuli being used to study auditory function in awake, behaving primate experiments. We show that principal-component analysis extracts features that are closely related to the dominant Fourier components of the stimuli, often called formants in the study of speech perception. Conversely, independent-component analysis extracts features that preserve the relative phase across a set of harmonically related frequencies. We have used several statistical techniques to explore the original and filtered stimuli, as well as the components extracted by each technique. This novel approach provides a powerful method for determining the essential features within complex stimuli that activate higher-order sensory neurons.