A hierarchy of processing complexity and timescales for natural sounds in human auditory cortex.

Efficient behavior is supported by humans' ability to rapidly recognize acoustically distinct sounds as members of a common category. Within auditory cortex, there are critical unanswered questions regarding the organization and dynamics of sound categorization. Here, we performed intracerebral recordings in the context of epilepsy surgery as 20 patient-participants listened to natural sounds. We built encoding models to predict neural responses using features of these sounds extracted from different layers within a sound-categorization deep neural network (DNN). This approach yielded highly accurate models of neural responses throughout auditory cortex. The complexity of a cortical site's representation (measured by the depth of the DNN layer that produced the best model) was closely related to its anatomical location, with shallow, middle, and deep layers of the DNN associated with core (primary auditory cortex), lateral belt, and parabelt regions, respectively. Smoothly varying gradients of representational complexity also existed within these regions, with complexity increasing along a posteromedial-to-anterolateral direction in core and lateral belt, and along posterior-to-anterior and dorsal-to-ventral dimensions in parabelt. When we estimated the time window over which each recording site integrates information, we found shorter integration windows in core relative to lateral belt and parabelt. Lastly, we found a relationship between the length of the integration window and the complexity of information processing within core (but not lateral belt or parabelt). These findings suggest hierarchies of timescales and processing complexity, and their interrelationship, represent a functional organizational principle of the auditory stream that underlies our perception of complex, abstract auditory information.

Electrocorticography reveals the dynamics of famous voice responses in human fusiform gyrus.

Voice and face processing occur through convergent neural systems that facilitate speaker recognition. Neuroimaging studies suggest that familiar voice processing engages early visual cortex, including the bilateral fusiform gyrus (FG) on the basal temporal lobe. However, what role the FG plays in voice processing and whether it is driven by bottom-up or top-down mechanisms is unresolved. In this study we directly examined neural responses to famous voices and faces in human FG with direct cortical surface recordings (electrocorticography) in epilepsy surgery patients. We tested the hypothesis that neural populations in human FG respond to famous voices and investigated the temporal properties of voice responses in FG. Recordings were acquired from five adult participants during a person identification task using visual and auditory stimuli from famous speakers (U.S. Presidents Barack Obama, George W. Bush, and Bill Clinton). Patients were presented with images of presidents or clips of their voices and asked to identify the portrait/speaker. Our results demonstrate that a subset of face-responsive sites in and near FG also exhibit voice responses that are both lower in magnitude and delayed (300-600 ms) compared with visual responses. The dynamics of voice processing revealed by direct cortical recordings suggests a top-down feedback-mediated response to famous voices in FG that may facilitate speaker identification.NEW & NOTEWORTHY Interactions between auditory and visual cortices play an important role in person identification, but the dynamics of these interactions remain poorly understood. We performed direct brain recordings of fusiform face cortex in human epilepsy patients performing a famous voice naming task, revealing the dynamics of famous voice processing in human fusiform face cortex. The findings support a model of top-down interactions from auditory to visual cortex to facilitate famous voice recognition.

Neural responses in human superior temporal cortex support coding of voice representations.

The ability to recognize abstract features of voice during auditory perception is an intricate feat of human audition. For the listener, this occurs in near-automatic fashion to seamlessly extract complex cues from a highly variable auditory signal. Voice perception depends on specialized regions of auditory cortex, including superior temporal gyrus (STG) and superior temporal sulcus (STS). However, the nature of voice encoding at the cortical level remains poorly understood. We leverage intracerebral recordings across human auditory cortex during presentation of voice and nonvoice acoustic stimuli to examine voice encoding at the cortical level in 8 patient-participants undergoing epilepsy surgery evaluation. We show that voice selectivity increases along the auditory hierarchy from supratemporal plane (STP) to the STG and STS. Results show accurate decoding of vocalizations from human auditory cortical activity even in the complete absence of linguistic content. These findings show an early, less-selective temporal window of neural activity in the STG and STS followed by a sustained, strongly voice-selective window. Encoding models demonstrate divergence in the encoding of acoustic features along the auditory hierarchy, wherein STG/STS responses are best explained by voice category and acoustics, as opposed to acoustic features of voice stimuli alone. This is in contrast to neural activity recorded from STP, in which responses were accounted for by acoustic features. These findings support a model of voice perception that engages categorical encoding mechanisms within STG and STS to facilitate feature extraction.