The neurophysiology of closed-loop auditory stimulation in sleep: A magnetoencephalography study.

Closed-loop auditory stimulation (CLAS) is a brain modulation technique in which sounds are timed to enhance or disrupt endogenous neurophysiological events. CLAS of slow oscillation up-states in sleep is becoming a popular tool to study and enhance sleep's functions, as it increases slow oscillations, evokes sleep spindles and enhances memory consolidation of certain tasks. However, few studies have examined the specific neurophysiological mechanisms involved in CLAS, in part because of practical limitations to available tools. To evaluate evidence for possible models of how sound stimulation during brain up-states alters brain activity, we simultaneously recorded electro- and magnetoencephalography in human participants who received auditory stimulation across sleep stages. We conducted a series of analyses that test different models of pathways through which CLAS of slow oscillations may affect widespread neural activity that have been suggested in literature, using spatial information, timing and phase relationships in the source-localized magnetoencephalography data. The results suggest that auditory information reaches ventral frontal lobe areas via non-lemniscal pathways. From there, a slow oscillation is created and propagated. We demonstrate that while the state of excitability of tissue in auditory cortex and frontal ventral regions shows some synchrony with the electroencephalography (EEG)-recorded up-states that are commonly used for CLAS, it is the state of ventral frontal regions that is most critical for slow oscillation generation. Our findings advance models of how CLAS leads to enhancement of slow oscillations, sleep spindles and associated cognitive benefits and offer insight into how the effectiveness of brain stimulation techniques can be improved.

Approaches to studying emotion using physiological responses to spoken narratives: A scoping review.

Narratives are effective tools for evoking emotions, and physiological measurements provide a means of objectively assessing emotional reactions - making them a potentially powerful pair of tools for studying emotional processes. However, extent research combining emotional narratives and physiological measurement varies widely in design and application, making it challenging to identify previous work, consolidate findings, and design effective experiments. Our scoping review explores the use of auditory emotional narratives and physiological measures in research, examining paradigms, study populations, and represented emotions. Following the PRISMA-ScR Checklist, we searched five databases for peer-reviewed experimental studies that used spoken narratives to induce emotion and reported autonomic physiological measures. Among 3466 titles screened and 653 articles reviewed, 110 studies were included. Our exploration revealed a variety of applications and experimental paradigms; emotional narratives paired with physiological measures have been used to study diverse topics and populations, including neurotypical and clinical groups. Although incomparable designs and sometimes contradictory results precluded general recommendations as regards which physiological measures to use when designing new studies, as a whole, the body of work suggests that these tools can be valuable to study emotions. Our review offers an overview of research employing narratives and physiological measures for emotion study, and highlights weaknesses in reporting practices and gaps in our knowledge concerning the robustness and specificity of physiological measures as indices of emotion. We discuss study design considerations and transparent reporting, to facilitate future using emotional narratives and physiological measures in studying emotions.

Oscillatory Entrainment of the Frequency-following Response in Auditory Cortical and Subcortical Structures.

There is much debate about the existence and function of neural oscillatory mechanisms in the auditory system. The frequency-following response (FFR) is an index of neural periodicity encoding that can provide a vehicle to study entrainment in frequency ranges relevant to speech and music processing. Criteria for entrainment include the presence of poststimulus oscillations and phase alignment between stimulus and endogenous activity. To test the hypothesis of entrainment, in experiment 1 we collected FFR data for a repeated syllable using magnetoencephalography (MEG) and electroencephalography in 20 male and female human adults. We observed significant oscillatory activity after stimulus offset in auditory cortex and subcortical auditory nuclei, consistent with entrainment. In these structures, the FFR fundamental frequency converged from a lower value over 100 ms to the stimulus frequency, consistent with phase alignment, and diverged to a lower value after offset, consistent with relaxation to a preferred frequency. In experiment 2, we tested how transitions between stimulus frequencies affected the MEG FFR to a train of tone pairs in 30 people. We found that the FFR was affected by the frequency of the preceding tone for up to 40 ms at subcortical levels, and even longer durations at cortical levels. Our results suggest that oscillatory entrainment may be an integral part of periodic sound representation throughout the auditory neuraxis. The functional role of this mechanism is unknown, but it could serve as a fine-scale temporal predictor for frequency information, enhancing stability and reducing susceptibility to degradation that could be useful in real-life noisy environments.SIGNIFICANCE STATEMENT Neural oscillations are proposed to be a ubiquitous aspect of neural function, but their contribution to auditory encoding is not clear, particularly at higher frequencies associated with pitch encoding. In a magnetoencephalography experiment, we found converging evidence that the frequency-following response has an oscillatory component according to established criteria: poststimulus resonance, progressive entrainment of the neural frequency to the stimulus frequency, and relaxation toward the original state on stimulus offset. In a second experiment, we found that the frequency and amplitude of the frequency-following response to tones are affected by preceding stimuli. These findings support the contribution of intrinsic oscillations to the encoding of sound, and raise new questions about their functional roles, possibly including stabilization and low-level predictive coding.

Cortical Correlates of the Auditory Frequency-Following and Onset Responses: EEG and fMRI Evidence.

The frequency-following response (FFR) is a measure of the brain's periodic sound encoding. It is of increasing importance for studying the human auditory nervous system due to numerous associations with auditory cognition and dysfunction. Although the FFR is widely interpreted as originating from brainstem nuclei, a recent study using MEG suggested that there is also a right-lateralized contribution from the auditory cortex at the fundamental frequency (Coffey et al., 2016b). Our objectives in the present work were to validate and better localize this result using a completely different neuroimaging modality and to document the relationships between the FFR, the onset response, and cortical activity. Using a combination of EEG, fMRI, and diffusion-weighted imaging, we show that activity in the right auditory cortex is related to individual differences in FFR-fundamental frequency (f0) strength, a finding that was replicated with two independent stimulus sets, with and without acoustic energy at the fundamental frequency. We demonstrate a dissociation between this FFR-f0-sensitive response in the right and an area in left auditory cortex that is sensitive to individual differences in the timing of initial response to sound onset. Relationships to timing and their lateralization are supported by parallels in the microstructure of the underlying white matter, implicating a mechanism involving neural conduction efficiency. These data confirm that the FFR has a cortical contribution and suggest ways in which auditory neuroscience may be advanced by connecting early sound representation to measures of higher-level sound processing and cognitive function. SIGNIFICANCE STATEMENT: The frequency-following response (FFR) is an EEG signal that is used to explore how the auditory system encodes temporal regularities in sound and is related to differences in auditory function between individuals. It is known that brainstem nuclei contribute to the FFR, but recent findings of an additional cortical source are more controversial. Here, we use fMRI to validate and extend the prediction from MEG data of a right auditory cortex contribution to the FFR. We also demonstrate a dissociation between FFR-related cortical activity from that related to the latency of the response to sound onset, which is found in left auditory cortex. The findings provide a clearer picture of cortical processes for analysis of sound features.

Dissociation of Neural Networks for Predisposition and for Training-Related Plasticity in Auditory-Motor Learning.

Skill learning results in changes to brain function, but at the same time individuals strongly differ in their abilities to learn specific skills. Using a 6-week piano-training protocol and pre- and post-fMRI of melody perception and imagery in adults, we dissociate learning-related patterns of neural activity from pre-training activity that predicts learning rates. Fronto-parietal and cerebellar areas related to storage of newly learned auditory-motor associations increased their response following training; in contrast, pre-training activity in areas related to stimulus encoding and motor control, including right auditory cortex, hippocampus, and caudate nuclei, was predictive of subsequent learning rate. We discuss the implications of these results for models of perceptual and of motor learning. These findings highlight the importance of considering individual predisposition in plasticity research and applications.

In vivo two-photon Ca2+ imaging reveals selective reward effects on stimulus-specific assemblies in mouse visual cortex.

Experiences can alter functional properties of neurons in primary sensory neocortex but it is poorly understood how stimulus-reward associations contribute to these changes. Using in vivo two-photon calcium imaging in mouse primary visual cortex (V1), we show that association of a directional visual stimulus with reward results in broadened orientation tuning and sharpened direction tuning in a stimulus-selective subpopulation of V1 neurons. Neurons with preferred orientations similar, but not identical to, the CS+ selectively increased their tuning curve bandwidth and thereby exhibited an increased response amplitude at the CS+ orientation. The increase in response amplitude was observed for a small range of orientations around the CS+ orientation. A nonuniform spatial distribution of reward effects across the cortical surface was observed, as the spatial distance between pairs of CS+ tuned neurons was reduced compared with pairs of CS- tuned neurons and pairs of control directions or orientations. These data show that, in primary visual cortex, formation of a stimulus-reward association results in selective alterations in stimulus-specific assemblies rather than population-wide effects.

Overcoming boundaries: Interdisciplinary challenges and opportunities in cognitive neuroscience.

Cognitive neuroscience has considerable untapped potential to translate our understanding of brain function into applications that maintain, restore, or enhance human cognition. Complex, real-world phenomena encountered in daily life, professional contexts, and in the arts, can also be a rich source of information for better understanding cognition, which in turn can lead to advances in knowledge and health outcomes. Interdisciplinary work is needed for these bi-directional benefits to be realized. Our cognitive neuroscience team has been collaborating on several interdisciplinary projects: hardware and software development for brain stimulation, measuring human operator state in safety-critical robotics environments, and exploring emotional regulation in actors who perform traumatic narratives. Our approach is to study research questions of mutual interest in the contexts of domain-specific applications, using (and sometimes improving) the experimental tools and techniques of cognitive neuroscience. These interdisciplinary attempts are described as case studies in the present work to illustrate non-trivial challenges that come from working across traditional disciplinary boundaries. We reflect on how obstacles to interdisciplinary work can be overcome, with the goals of enriching our understanding of human cognition and amplifying the positive effects cognitive neuroscientists have on society and innovation.

Specificity of Affective Responses in Misophonia Depends on Trigger Identification.

Individuals with misophonia, a disorder involving extreme sound sensitivity, report significant anger, disgust, and anxiety in response to select but usually common sounds. While estimates of prevalence within certain populations such as college students have approached 20%, it is currently unknown what percentage of people experience misophonic responses to such "trigger" sounds. Furthermore, there is little understanding of the fundamental processes involved. In this study, we aimed to characterize the distribution of misophonic symptoms in a general population, as well as clarify whether the aversive emotional responses to trigger sounds are partly caused by acoustic salience of the sound itself, or by recognition of the sound. Using multi-talker babble as masking noise to decrease participants' ability to identify sounds, we assessed how identification of common trigger sounds related to subjective emotional responses in 300 adults who participated in an online study. Participants were asked to listen to and identify neutral, unpleasant and trigger sounds embedded in different levels of the masking noise (signal-to-noise ratios: -30, -20, -10, 0, +10 dB), and then to evaluate their subjective judgment of the sounds (pleasantness) and emotional reactions to them (anxiety, anger, and disgust). Using participants' scores on a scale quantifying misophonia sensitivity, we selected the top and bottom 20% scorers from the distribution to form a Most-Misophonic subgroup (N = 66) and Least-Misophonic subgroup (N = 68). Both groups were better at identifying triggers than unpleasant sounds, which themselves were identified better than neutral sounds. Both groups also recognized the aversiveness of the unpleasant and trigger sounds, yet for the Most-Misophonic group, there was a greater increase in subjective ratings of negative emotions once the sounds became identifiable, especially for trigger sounds. These results highlight the heightened salience of trigger sounds, but furthermore suggest that learning and higher-order evaluation of sounds play an important role in misophonia.

Sleep affects higher-level categorization of speech sounds, but not frequency encoding.

Sleep can increase consolidation of new knowledge and skills. It is less clear whether sleep plays a role in other aspects of experience-dependent neuroplasticity, which underlie important human capabilities such as spoken language processing. Theories of sensory learning differ in their predictions; some imply rapid learning at early sensory levels, while other propose a slow, progressive timecourse such that higher-level categorical representations guide immediate, novice learning, while lower-level sensory changes do not emerge until later stages. In this study, we investigated the role of sleep across both behavioural and physiological indices of auditory neuroplasticity. Forty healthy young human adults (23 female) who did not speak a tonal language participated in the study. They learned to categorize non-native Mandarin lexical tones using a sound-to-category training paradigm, and were then randomly assigned to a Nap or Wake condition. Polysomnographic data were recorded to quantify sleep during a 3 h afternoon nap opportunity, or equivalent period of quiet wakeful activity. Measures of behavioural performance accuracy revealed a significant improvement in learning the sound-to-category training paradigm between Nap and Wake groups. Conversely, a neural index of fine sound encoding fidelity of speech sounds known as the frequency-following response (FFR) suggested no change due to sleep, and a null model was supported, using Bayesian statistics. Together, these results support theories that propose a slow, progressive and hierarchical timecourse for sensory learning. Sleep's effect may play the biggest role in the higher-level learning, although contributions to more protracted processes of plasticity that exceed the study duration cannot be ruled out.

The Portiloop: A deep learning-based open science tool for closed-loop brain stimulation.

Closed-loop brain stimulation refers to capturing neurophysiological measures such as electroencephalography (EEG), quickly identifying neural events of interest, and producing auditory, magnetic or electrical stimulation so as to interact with brain processes precisely. It is a promising new method for fundamental neuroscience and perhaps for clinical applications such as restoring degraded memory function; however, existing tools are expensive, cumbersome, and offer limited experimental flexibility. In this article, we propose the Portiloop, a deep learning-based, portable and low-cost closed-loop stimulation system able to target specific brain oscillations. We first document open-hardware implementations that can be constructed from commercially available components. We also provide a fast, lightweight neural network model and an exploration algorithm that automatically optimizes the model hyperparameters to the desired brain oscillation. Finally, we validate the technology on a challenging test case of real-time sleep spindle detection, with results comparable to off-line expert performance on the Massive Online Data Annotation spindle dataset (MODA; group consensus). Software and plans are available to the community as an open science initiative to encourage further development and advance closed-loop neuroscience research [https://github.com/Portiloop].

Decoding of Envelope vs. Fundamental Frequency During Complex Auditory Stream Segregation.

Hearing-in-noise perception is a challenging task that is critical to human function, but how the brain accomplishes it is not well understood. A candidate mechanism proposes that the neural representation of an attended auditory stream is enhanced relative to background sound via a combination of bottom-up and top-down mechanisms. To date, few studies have compared neural representation and its task-related enhancement across frequency bands that carry different auditory information, such as a sound's amplitude envelope (i.e., syllabic rate or rhythm; 1-9 Hz), and the fundamental frequency of periodic stimuli (i.e., pitch; >40 Hz). Furthermore, hearing-in-noise in the real world is frequently both messier and richer than the majority of tasks used in its study. In the present study, we use continuous sound excerpts that simultaneously offer predictive, visual, and spatial cues to help listeners separate the target from four acoustically similar simultaneously presented sound streams. We show that while both lower and higher frequency information about the entire sound stream is represented in the brain's response, the to-be-attended sound stream is strongly enhanced only in the slower, lower frequency sound representations. These results are consistent with the hypothesis that attended sound representations are strengthened progressively at higher level, later processing stages, and that the interaction of multiple brain systems can aid in this process. Our findings contribute to our understanding of auditory stream separation in difficult, naturalistic listening conditions and demonstrate that pitch and envelope information can be decoded from single-channel EEG data.

The Music-In-Noise Task (MINT): A Tool for Dissecting Complex Auditory Perception.

The ability to segregate target sounds in noisy backgrounds is relevant both to neuroscience and to clinical applications. Recent research suggests that hearing-in-noise (HIN) problems are solved using combinations of sub-skills that are applied according to task demand and information availability. While evidence is accumulating for a musician advantage in HIN, the exact nature of the reported training effect is not fully understood. Existing HIN tests focus on tasks requiring understanding of speech in the presence of competing sound. Because visual, spatial and predictive cues are not systematically considered in these tasks, few tools exist to investigate the most relevant components of cognitive processes involved in stream segregation. We present the Music-In-Noise Task (MINT) as a flexible tool to expand HIN measures beyond speech perception, and for addressing research questions pertaining to the relative contributions of HIN sub-skills, inter-individual differences in their use, and their neural correlates. The MINT uses a match-mismatch trial design: in four conditions (Baseline, Rhythm, Spatial, and Visual) subjects first hear a short instrumental musical excerpt embedded in an informational masker of "multi-music" noise, followed by either a matching or scrambled repetition of the target musical excerpt presented in silence; the four conditions differ according to the presence or absence of additional cues. In a fifth condition (Prediction), subjects hear the excerpt in silence as a target first, which helps to anticipate incoming information when the target is embedded in masking sound. Data from samples of young adults show that the MINT has good reliability and internal consistency, and demonstrate selective benefits of musicianship in the Prediction, Rhythm, and Visual subtasks. We also report a performance benefit of multilingualism that is separable from that of musicianship. Average MINT scores were correlated with scores on a sentence-in-noise perception task, but only accounted for a relatively small percentage of the variance, indicating that the MINT is sensitive to additional factors and can provide a complement and extension of speech-based tests for studying stream segregation. A customizable version of the MINT is made available for use and extension by the scientific community.

Evolving perspectives on the sources of the frequency-following response.

The auditory frequency-following response (FFR) is a non-invasive index of the fidelity of sound encoding in the brain, and is used to study the integrity, plasticity, and behavioral relevance of the neural encoding of sound. In this Perspective, we review recent evidence suggesting that, in humans, the FFR arises from multiple cortical and subcortical sources, not just subcortically as previously believed, and we illustrate how the FFR to complex sounds can enhance the wider field of auditory neuroscience. Far from being of use only to study basic auditory processes, the FFR is an uncommonly multifaceted response yielding a wealth of information, with much yet to be tapped.

Expedition Cognition: A Review and Prospective of Subterranean Neuroscience With Spaceflight Applications.

Renewed interest in human space exploration has highlighted the gaps in knowledge needed for successful long-duration missions outside low-Earth orbit. Although the technical challenges of such missions are being systematically overcome, many of the unknowns in predicting mission success depend on human behavior and performance, knowledge of which must be either obtained through space research or extrapolated from human experience on Earth. Particularly in human neuroscience, laboratory-based research efforts are not closely connected to real environments such as human space exploration. As caves share several of the physical and psychological challenges of spaceflight, underground expeditions have recently been developed as a spaceflight analog for astronaut training purposes, suggesting that they might also be suitable for studying aspects of behavior and cognition that cannot be fully examined under laboratory conditions. Our objective is to foster a bi-directional exchange between cognitive neuroscientists and expedition experts by (1) describing the cave environment as a worthy space analog for human research, (2) reviewing work conducted on human neuroscience and cognition within caves, (3) exploring the range of topics for which the unique environment may prove valuable as well as obstacles and limitations, (4) outlining technologies and methods appropriate for cave use, and (5) suggesting how researchers might establish contact with potential expedition collaborators. We believe that cave expeditions, as well as other sorts of expeditions, offer unique possibilities for cognitive neuroscience that will complement laboratory work and help to improve human performance and safety in operational environments, both on Earth and in space.

Speech-in-noise perception in musicians: A review.

The ability to understand speech in the presence of competing sound sources is an important neuroscience question in terms of how the nervous system solves this computational problem. It is also a critical clinical problem that disproportionally affects the elderly, children with language-related learning disorders, and those with hearing loss. Recent evidence that musicians have an advantage on this multifaceted skill has led to the suggestion that musical training might be used to improve or delay the decline of speech-in-noise (SIN) function. However, enhancements have not been universally reported, nor have the relative contributions of different bottom-up versus top-down processes, and their relation to preexisting factors been disentangled. This information that would be helpful to establish whether there is a real effect of experience, what exactly is its nature, and how future training-based interventions might target the most relevant components of cognitive processes. These questions are complicated by important differences in study design and uneven coverage of neuroimaging modality. In this review, we aim to systematize recent results from studies that have specifically looked at musician-related differences in SIN by their study design properties, to summarize the findings, and to identify knowledge gaps for future work.

Neural Correlates of Early Sound Encoding and their Relationship to Speech-in-Noise Perception.

Speech-in-noise (SIN) perception is a complex cognitive skill that affects social, vocational, and educational activities. Poor SIN ability particularly affects young and elderly populations, yet varies considerably even among healthy young adults with normal hearing. Although SIN skills are known to be influenced by top-down processes that can selectively enhance lower-level sound representations, the complementary role of feed-forward mechanisms and their relationship to musical training is poorly understood. Using a paradigm that minimizes the main top-down factors that have been implicated in SIN performance such as working memory, we aimed to better understand how robust encoding of periodicity in the auditory system (as measured by the frequency-following response) contributes to SIN perception. Using magnetoencephalograpy, we found that the strength of encoding at the fundamental frequency in the brainstem, thalamus, and cortex is correlated with SIN accuracy. The amplitude of the slower cortical P2 wave was previously also shown to be related to SIN accuracy and FFR strength; we use MEG source localization to show that the P2 wave originates in a temporal region anterior to that of the cortical FFR. We also confirm that the observed enhancements were related to the extent and timing of musicianship. These results are consistent with the hypothesis that basic feed-forward sound encoding affects SIN perception by providing better information to later processing stages, and that modifying this process may be one mechanism through which musical training might enhance the auditory networks that subserve both musical and language functions.

Individual Differences in the Frequency-Following Response: Relation to Pitch Perception.

The scalp-recorded frequency-following response (FFR) is a measure of the auditory nervous system's representation of periodic sound, and may serve as a marker of training-related enhancements, behavioural deficits, and clinical conditions. However, FFRs of healthy normal subjects show considerable variability that remains unexplained. We investigated whether the FFR representation of the frequency content of a complex tone is related to the perception of the pitch of the fundamental frequency. The strength of the fundamental frequency in the FFR of 39 people with normal hearing was assessed when they listened to complex tones that either included or lacked energy at the fundamental frequency. We found that the strength of the fundamental representation of the missing fundamental tone complex correlated significantly with people's general tendency to perceive the pitch of the tone as either matching the frequency of the spectral components that were present, or that of the missing fundamental. Although at a group level the fundamental representation in the FFR did not appear to be affected by the presence or absence of energy at the same frequency in the stimulus, the two conditions were statistically distinguishable for some subjects individually, indicating that the neural representation is not linearly dependent on the stimulus content. In a second experiment using a within-subjects paradigm, we showed that subjects can learn to reversibly select between either fundamental or spectral perception, and that this is accompanied both by changes to the fundamental representation in the FFR and to cortical-based gamma activity. These results suggest that both fundamental and spectral representations coexist, and are available for later auditory processing stages, the requirements of which may also influence their relative strength and thus modulate FFR variability. The data also highlight voluntary mode perception as a new paradigm with which to study top-down vs bottom-up mechanisms that support the emerging view of the FFR as the outcome of integrated processing in the entire auditory system.

Cortical contributions to the auditory frequency-following response revealed by MEG.

The auditory frequency-following response (FFR) to complex periodic sounds is used to study the subcortical auditory system, and has been proposed as a biomarker for disorders that feature abnormal sound processing. Despite its value in fundamental and clinical research, the neural origins of the FFR are unclear. Using magnetoencephalography, we observe a strong, right-asymmetric contribution to the FFR from the human auditory cortex at the fundamental frequency of the stimulus, in addition to signal from cochlear nucleus, inferior colliculus and medial geniculate. This finding is highly relevant for our understanding of plasticity and pathology in the auditory system, as well as higher-level cognition such as speech and music processing. It suggests that previous interpretations of the FFR may need re-examination using methods that allow for source separation.

Task decomposition: a framework for comparing diverse training models in human brain plasticity studies.

Training studies, in which the structural or functional neurophysiology is compared before and after expertise is acquired, are increasingly being used as models for understanding the human brain's potential for reorganization. It is proving difficult to use these results to answer basic and important questions like how task training leads to both specific and general changes in behavior and how these changes correspond with modifications in the brain. The main culprit is the diversity of paradigms used as complex task models. An assortment of activities ranging from juggling to deciphering Morse code has been reported. Even when working in the same general domain, few researchers use similar training models. New ways to meaningfully compare complex tasks are needed. We propose a method for characterizing and deconstructing the task requirements of complex training paradigms, which is suitable for application to both structural and functional neuroimaging studies. We believe this approach will aid brain plasticity research by making it easier to compare training paradigms, identify "missing puzzle pieces," and encourage researchers to design training protocols to bridge these gaps.