Hammerstein-Wiener Motion Artifact Correction for Functional Near-Infrared Spectroscopy: A Novel Inertial Measurement Unit-Based Technique.

Participant movement is a major source of artifacts in functional near-infrared spectroscopy (fNIRS) experiments. Mitigating the impact of motion artifacts (MAs) is crucial to estimate brain activity robustly. Here, we suggest and evaluate a novel application of the nonlinear Hammerstein-Wiener model to estimate and mitigate MAs in fNIRS signals from direct-movement recordings through IMU sensors mounted on the participant's head (head-IMU) and the fNIRS probe (probe-IMU). To this end, we analyzed the hemodynamic responses of single-channel oxyhemoglobin (HbO) and deoxyhemoglobin (HbR) signals from 17 participants who performed a hand tapping task with different levels of concurrent head movement. Additionally, the tapping task was performed without head movements to estimate the ground-truth brain activation. We compared the performance of our novel approach with the probe-IMU and head-IMU to eight established methods (PCA, tPCA, spline, spline Savitzky-Golay, wavelet, CBSI, RLOESS, and WCBSI) on four quality metrics: SNR, △AUC, RMSE, and R. Our proposed nonlinear Hammerstein-Wiener method achieved the best SNR increase (p < 0.001) among all methods. Visual inspection revealed that our approach mitigated MA contaminations that other techniques could not remove effectively. MA correction quality was comparable with head- and probe-IMUs.

GAUDIE: Development, validation, and exploration of a naturalistic German AUDItory Emotional database.

Since thoroughly validated naturalistic affective German speech stimulus databases are rare, we present here a novel validated database of speech sequences assembled with the purpose of emotion induction. The database comprises 37 audio speech sequences with a total duration of 92 minutes for the induction of positive, neutral, and negative emotion: comedian shows intending to elicit humorous and amusing feelings, weather forecasts, and arguments between couples and relatives from movies or television series. Multiple continuous and discrete ratings are used to validate the database to capture the time course and variabilities of valence and arousal. We analyse and quantify how well the audio sequences fulfil quality criteria of differentiation, salience/strength, and generalizability across participants. Hence, we provide a validated speech database of naturalistic scenarios suitable to investigate emotion processing and its time course with German-speaking participants. Information on using the stimulus database for research purposes can be found at the OSF project repository GAUDIE: https://osf.io/xyr6j/ .

AI-assisted ethics? considerations of AI simulation for the ethical assessment and design of assistive technologies.

Current ethical debates on the use of artificial intelligence (AI) in healthcare treat AI as a product of technology in three ways. First, by assessing risks and potential benefits of currently developed AI-enabled products with ethical checklists; second, by proposing ex ante lists of ethical values seen as relevant for the design and development of assistive technology, and third, by promoting AI technology to use moral reasoning as part of the automation process. The dominance of these three perspectives in the discourse is demonstrated by a brief summary of the literature. Subsequently, we propose a fourth approach to AI, namely, as a methodological tool to assist ethical reflection. We provide a concept of an AI-simulation informed by three separate elements: 1) stochastic human behavior models based on behavioral data for simulating realistic settings, 2) qualitative empirical data on value statements regarding internal policy, and 3) visualization components that aid in understanding the impact of changes in these variables. The potential of this approach is to inform an interdisciplinary field about anticipated ethical challenges or ethical trade-offs in concrete settings and, hence, to spark a re-evaluation of design and implementation plans. This may be particularly useful for applications that deal with extremely complex values and behavior or with limitations on the communication resources of affected persons (e.g., persons with dementia care or for care of persons with cognitive impairment). Simulation does not replace ethical reflection but does allow for detailed, context-sensitive analysis during the design process and prior to implementation. Finally, we discuss the inherently quantitative methods of analysis afforded by stochastic simulations as well as the potential for ethical discussions and how simulations with AI can improve traditional forms of thought experiments and future-oriented technology assessment.

Improved Motion Artifact Correction in fNIRS Data by Combining Wavelet and Correlation-Based Signal Improvement.

Functional near-infrared spectroscopy (fNIRS) is an optical non-invasive neuroimaging technique that allows participants to move relatively freely. However, head movements frequently cause optode movements relative to the head, leading to motion artifacts (MA) in the measured signal. Here, we propose an improved algorithmic approach for MA correction that combines wavelet and correlation-based signal improvement (WCBSI). We compare its MA correction accuracy to multiple established correction approaches (spline interpolation, spline-Savitzky-Golay filter, principal component analysis, targeted principal component analysis, robust locally weighted regression smoothing filter, wavelet filter, and correlation-based signal improvement) on real data. Therefore, we measured brain activity in 20 participants performing a hand-tapping task and simultaneously moving their head to produce MAs at different levels of severity. In order to obtain a "ground truth" brain activation, we added a condition in which only the tapping task was performed. We compared the MA correction performance among the algorithms on four predefined metrics (R, RMSE, MAPE, and ΔAUC) and ranked the performances. The suggested WCBSI algorithm was the only one exceeding average performance (p < 0.001), and it had the highest probability to be the best ranked algorithm (78.8% probability). Together, our results indicate that among all algorithms tested, our suggested WCBSI approach performed consistently favorably across all measures.

FAIR human neuroscientific data sharing to advance AI driven research and applications: Legal frameworks and missing metadata standards.

Modern AI supported research holds many promises for basic and applied science. However, the application of AI methods is often limited because most labs cannot, on their own, acquire large and diverse datasets, which are best for training these methods. Data sharing and open science initiatives promise some relief to the problem, but only if the data are provided in a usable way. The FAIR principles state very general requirements for useful data sharing: they should be findable, accessible, interoperable, and reusable. This article will focus on two challenges to implement the FAIR framework for human neuroscience data. On the one hand, human data can fall under special legal protection. The legal frameworks regulating how and what data can be openly shared differ greatly across countries which can complicate data sharing or even discourage researchers from doing so. Moreover, openly accessible data require standardization of data and metadata organization and annotation in order to become interpretable and useful. This article briefly introduces open neuroscience initiatives that support the implementation of the FAIR principles. It then reviews legal frameworks, their consequences for accessibility of human neuroscientific data and some ethical implications. We hope this comparison of legal jurisdictions helps to elucidate that some alleged obstacles for data sharing only require an adaptation of procedures but help to protect the privacy of our most generous donors to research … our study participants. Finally, it elaborates on the problem of missing standards for metadata annotation and introduces initiatives that aim at developing tools to make neuroscientific data acquisition and analysis pipelines FAIR by design. While the paper focuses on making human neuroscience data useful for data-intensive AI the general considerations hold for other fields where large amounts of openly available human data would be helpful.

Inter-individual single-trial classification of MEG data using M-CCA.

Neuroscientific studies often involve some form of group analysis over multiple participants. This requires alignment of recordings across participants. A naive solution is to assume that participants' recordings can be aligned anatomically in sensor space. However, this assumption is likely violated due to anatomical and functional differences between individual brains. In magnetoencephalography (MEG) recordings the problem of inter-subject alignment is exacerbated by the susceptibility of MEG to individual cortical folding patterns as well as the inter-subject variability of sensor locations over the brain due to the use of a fixed helmet. Hence, an approach to combine MEG data over individual brains should relax the assumptions that a) brain anatomy and function are tightly linked and b) that the same sensors capture functionally comparable brain activation across individuals. Here we use multiset canonical correlation analysis (M-CCA) to find a common representation of MEG activations recorded from 15 participants performing a grasping task. The M-CCA algorithm was applied to transform the data of a set of multiple participants into a common space with maximum correlation between participants. Importantly, we derive a method to transform data from a new, previously unseen participant into this common representation. This makes it useful for applications that require transfer of models derived from a group of individuals to new individuals. We demonstrate the usefulness and superiority of the approach with respect to previously used approaches. Finally, we show that our approach requires only a small number of labeled data from the new participant. The proposed method demonstrates that functionally motivated common spaces have potential applications in reducing training time of online brain-computer interfaces, where models can be pre-trained on previous participants/sessions. Moreover, inter-subject alignment via M-CCA has the potential for combining data of different participants and could become helpful in future endeavors on large open datasets.

Grasp-specific high-frequency broadband mirror neuron activity during reach-and-grasp movements in humans.

Broadly congruent mirror neurons, responding to any grasp movement, and strictly congruent mirror neurons, responding only to specific grasp movements, have been reported in single-cell studies with primates. Delineating grasp properties in humans is essential to understand the human mirror neuron system with implications for behavior and social cognition. We analyzed electrocorticography data from a natural reach-and-grasp movement observation and delayed imitation task with 3 different natural grasp types of everyday objects. We focused on the classification of grasp types from high-frequency broadband mirror activation patterns found in classic mirror system areas, including sensorimotor, supplementary motor, inferior frontal, and parietal cortices. Classification of grasp types was successful during movement observation and execution intervals but not during movement retention. Our grasp type classification from combined and single mirror electrodes provides evidence for grasp-congruent activity in the human mirror neuron system potentially arising from strictly congruent mirror neurons.

Multitasking While Driving: Central Bottleneck or Problem State Interference?

OBJECTIVE: The objective of this work was to investigate if visuospatial attention and working memory load interact at a central control resource or at a task-specific, information processing resource during driving. BACKGROUND: In previous multitasking driving experiments, interactions between different cognitive concepts (e.g., attention and working memory) have been found. These interactions have been attributed to a central bottleneck or to the so-called problem-state bottleneck, related to working memory usage. METHOD: We developed two different cognitive models in the cognitive architecture ACT-R, which implement the central vs. problem-state bottleneck. The models performed a driving task, during which we varied visuospatial attention and working memory load. We evaluated the model by conducting an experiment with human participants and compared the behavioral data to the model's behavior. RESULTS: The problem-state-bottleneck model could account for decreased driving performance due to working memory load as well as increased visuospatial attentional demands as compared to the central-bottleneck model, which could not account for effects of increased working memory load. CONCLUSION: The interaction between working memory and visuospatial attention in our dual tasking experiment can be best characterized by a bottleneck in the working memory. The model results suggest that as working memory load becomes higher, drivers manage to perform fewer control actions, which leads to decreasing driving performance. APPLICATION: Predictions about the effect of different mental loads can be used to quantify the contribution of each subtask allowing for precise assessments of the current overall mental load, which automated driving systems may adapt to.

Open and reproducible neuroimaging: From study inception to publication.

Empirical observations of how labs conduct research indicate that the adoption rate of open practices for transparent, reproducible, and collaborative science remains in its infancy. This is at odds with the overwhelming evidence for the necessity of these practices and their benefits for individual researchers, scientific progress, and society in general. To date, information required for implementing open science practices throughout the different steps of a research project is scattered among many different sources. Even experienced researchers in the topic find it hard to navigate the ecosystem of tools and to make sustainable choices. Here, we provide an integrated overview of community-developed resources that can support collaborative, open, reproducible, replicable, robust and generalizable neuroimaging throughout the entire research cycle from inception to publication and across different neuroimaging modalities. We review tools and practices supporting study inception and planning, data acquisition, research data management, data processing and analysis, and research dissemination. An online version of this resource can be found at https://oreoni.github.io. We believe it will prove helpful for researchers and institutions to make a successful and sustainable move towards open and reproducible science and to eventually take an active role in its future development.

Sublexical cues affect degraded speech processing: insights from fMRI.

In natural listening situations, speech perception is often impaired by degraded speech sounds arriving at the ear. Contextual speech information can improve the perception of degraded speech and modify neuronal responses elicited by degraded speech. However, most studies on context effects on neural responses to degraded speech confounded lexico-semantic and sublexical cues. Here, we used fMRI to investigate how prior sublexical speech (e.g. pseudowords cues) affects neural responses to degraded sublexical speech and hence its processing and recognition. Each trial consisted of three consecutively presented pseudowords, of which the first and third were identical and degraded. The second pseudoword was always presented in clear form and either matched or did not match the degraded pseudowords. Improved speech processing through sublexical processing was associated with BOLD activation increases in frontal, temporal, and parietal regions, including the primary auditory cortex (PAC), posterior superior temporal cortex, angular gyrus, supramarginal gyrus, middle temporal cortex, and somato-motor cortex. These brain regions are part of a speech processing network and are involved in lexico-semantic processing. To further investigate the adaptive changes in PAC, we conducted a bilateral region of interest analysis on PAC subregions. PAC ROIs showed bilaterally increased activation in the match condition compared with the mismatch condition. Our results show that the perception of unintelligible degraded speech is improved and the neuronal population response is enhanced after exposure to intact sublexical cues. Furthermore, our findings indicate that the processing of clear meaningless sublexical speech preceding degraded speech could enhance the activity in the brain regions that belong to the cortical speech processing network previously reported in studies investigating lexico-semantic speech.

Driver's turning intent recognition model based on brain activation and contextual information.

Traffic situations like turning at intersections are destined for safety-critical situations and accidents. Human errors are one of the main reasons for accidents in these situations. A model that recognizes the driver's turning intent could help to reduce accidents by warning the driver or stopping the vehicle before a dangerous turning maneuver. Most models that aim at predicting the probability of a driver's turning intent use only contextual information, such as gap size or waiting time. The objective of this study is to investigate whether the combination of context information and brain activation measurements enhances the recognition of turning intent. We conducted a driving simulator study while simultaneously measuring brain activation using high-density fNIRS. A neural network model for turning intent recognition was trained on the fNIRS and contextual data. The input variables were analyzed using SHAP (SHapley Additive exPlanations) feature importance analysis to show the positive effect of the inclusion of brain activation data. Both the model's evaluation and the feature importance analysis suggest that the combination of context information and brain activation leads to an improved turning intent recognition. The fNIRS results showed increased brain activation differences during the "turn" decision-making phase before turning execution in parts of the left motor cortices, such as the primary motor cortex (PMC; putative BA 4), premotor area (PMA; putative BA 6), and supplementary motor area (SMA; putative BA 8). Furthermore, we also observed increased activation differences in the left prefrontal areas, potentially in the left middle frontal gyrus (putative BA 9), which has been associated with the control of executive functions, such as decision-making and action planning. We hypothesize that brain activation measurements could be a more direct indicator with potentially high specificity for the turning behavior and thus help to increase the recognition model's performance.

Investigating Differences in Behavior and Brain in Human-Human and Human-Autonomous Vehicle Interactions in Time-Critical Situations.

Some studies provide evidence that humans could actively exploit the alleged technological advantages of autonomous vehicles (AVs). This implies that humans may tend to interact differently with AVs as compared to human driven vehicles (HVs) with the knowledge that AVs are programmed to be risk-averse. Hence, it is important to investigate how humans interact with AVs in complex traffic situations. Here, we investigated whether participants would value interactions with AVs differently compared to HVs, and if these differences can be characterized on the behavioral and brain-level. We presented participants with a cover story while recording whole-head brain activity using fNIRS that they were driving under time pressure through urban traffic in the presence of other HVs and AVs. Moreover, the AVs were programmed defensively to avoid collisions and had faster braking reaction times than HVs. Participants would receive a monetary reward if they managed to finish the driving block within a given time-limit without risky driving maneuvers. During the drive, participants were repeatedly confronted with left-lane turning situations at unsignalized intersections. They had to stop and find a gap to turn in front of an oncoming stream of vehicles consisting of HVs and AVs. While the behavioral results did not show any significant difference between the safety margin used during the turning maneuvers with respect to AVs or HVs, participants tended to be more certain in their decision-making process while turning in front of AVs as reflected by the smaller variance in the gap size acceptance as compared to HVs. Importantly, using a multivariate logistic regression approach, we were able to predict whether the participants decided to turn in front of HVs or AVs from whole-head fNIRS in the decision-making phase for every participant (mean accuracy = 67.2%, SD = 5%). Channel-wise univariate fNIRS analysis revealed increased brain activation differences for turning in front of AVs compared to HVs in brain areas that represent the valuation of actions taken during decision-making. The insights provided here may be useful for the development of control systems to assess interactions in future mixed traffic environments involving AVs and HVs.

Opportunities and Limitations of a Gaze-Contingent Display to Simulate Visual Field Loss in Driving Simulator Studies.

Background: Research on task performance under visual field loss is often limited due to small and heterogenous samples. Simulations of visual impairments hold the potential to account for many of those challenges. Digitally altered pictures, glasses, and contact lenses with partial occlusions have been used in the past. One of the most promising methods is the use of a gaze-contingent display that occludes parts of the visual field according to the current gaze position. In this study, the gaze-contingent paradigm was implemented in a static driving simulator to simulate visual field loss and to evaluate parallels in the resulting driving and gaze behavior in comparison to patients. Methods: The sample comprised 15 participants without visual impairment. All the subjects performed three drives: with full vision, simulated left-sided homonymous hemianopia, and simulated right-sided homonymous hemianopia, respectively. During each drive, the participants drove through an urban environment where they had to maneuver through intersections by crossing straight ahead, turning left, and turning right. Results: The subjects reported reduced safety and increased workload levels during simulated visual field loss, which was reflected in reduced lane position stability and greater absence of large gaze movements. Initial compensatory strategies could be found concerning a dislocated gaze position and a distorted fixation ratio toward the blind side, which was more pronounced for right-sided visual field loss. During left-sided visual field loss, the participants showed a smaller horizontal range of gaze positions, longer fixation durations, and smaller saccadic amplitudes compared to right-sided homonymous hemianopia and, more distinctively, compared to normal vision. Conclusion: The results largely mirror reports from driving and visual search tasks under simulated and pathological homonymous hemianopia concerning driving and scanning challenges, initially adopted compensatory strategies, and driving safety. This supports the notion that gaze-contingent displays can be a useful addendum to driving simulator research with visual impairments if the results are interpreted considering methodological limitations and inherent differences to the pathological impairment.

High-gamma mirror activity patterns in the human brain during reach-to-grasp movement observation, retention, and execution-An MEG study.

While the existence of a human mirror neuron system is evident, the involved brain areas and their exact functional roles remain under scientific debate. A number of functionally different mirror neuron types, neurons that selectively respond to specific grasp phases and types for example, have been reported with single cell recordings in monkeys. In humans, spatially limited, intracranially recorded electrophysiological signals in the high-gamma (HG) range have been used to investigate the human mirror system, as they are associated with spiking activity in single neurons. Our goal here is to complement previous intracranial HG studies by using magnetoencephalography to record HG activity simultaneously from the whole head. Participants performed a natural reach-to-grasp movement observation and delayed imitation task with different everyday objects and grasp types. This allowed us to characterize the spatial organization of cortical areas that show HG-activation modulation during movement observation (mirroring), retention (mnemonic mirroring), and execution (motor control). Our results show mirroring related HG modulation patterns over bilateral occipito-parietal as well as sensorimotor areas. In addition, we found mnemonic mirroring related HG modulation over contra-lateral fronto-temporal areas. These results provide a foundation for further human mirror system research as well as possible target areas for brain-computer interface and neurorehabilitation approaches.

Recommendations of Choice of Head Coil and Prescan Normalize Filter Depend on Region of Interest and Task.

The performance of MRI head coils together with the influence of the prescan normalize filter in different brain regions was evaluated. Functional and structural data were recorded from 26 participants performing motor, auditory, and visual tasks in different conditions: with the 20- and 64-channel Siemens head/neck coil and the prescan normalize filter turned ON or OFF. Data were analyzed with the MRIQC tool to evaluate data quality differences. The functional data were statistically evaluated by comparison of the ß estimates and the time-course signal-to-noise ratio (tSNR) in four regions of interest, i.e., the auditory, visual, and motor cortices and the thalamus. The MRIQC tool indicated a better data quality for both functional and structural data with the prescan normalize filter, with an advantage for the 20-channel head coil in functional data and an advantage for the 64-channel head coil in structural measurements. Nevertheless, recommendations for the functional data regarding choice of head coils and prescan normalize filter depend on the brain regions of interest. Higher ß estimates and tSNR values occurred in the auditory cortex and thalamus with the prescan normalize filter, whereas the contrary was true for the visual and motor cortices. Due to higher ß estimates in the visual cortex in the 64-channel head coil, this head coil is recommended for studies investigating the visual cortex. For most of the research questions, the 20-channel head coil is better suited for functional experiments, with the prescan normalize filter, especially when investigating deep brain areas. For anatomical studies, the 64-channel head coil seemed to be the better choice.

Generalizable dimensions of human cortical auditory processing of speech in natural soundscapes: A data-driven ultra high field fMRI approach.

Speech comprehension in natural soundscapes rests on the ability of the auditory system to extract speech information from a complex acoustic signal with overlapping contributions from many sound sources. Here we reveal the canonical processing of speech in natural soundscapes on multiple scales by using data-driven modeling approaches to characterize sounds to analyze ultra high field fMRI recorded while participants listened to the audio soundtrack of a movie. We show that at the functional level the neuronal processing of speech in natural soundscapes can be surprisingly low dimensional in the human cortex, highlighting the functional efficiency of the auditory system for a seemingly complex task. Particularly, we find that a model comprising three functional dimensions of auditory processing in the temporal lobes is shared across participants' fMRI activity. We further demonstrate that the three functional dimensions are implemented in anatomically overlapping networks that process different aspects of speech in natural soundscapes. One is most sensitive to complex auditory features present in speech, another to complex auditory features and fast temporal modulations, that are not specific to speech, and one codes mainly sound level. These results were derived with few a-priori assumptions and provide a detailed and computationally reproducible account of the cortical activity in the temporal lobe elicited by the processing of speech in natural soundscapes.

Effects of age-related hearing loss and hearing aid experience on sentence processing.

Age-related hearing loss typically affects the hearing of high frequencies in older adults. Such hearing loss influences the processing of spoken language, including higher-level processing such as that of complex sentences. Hearing aids may alleviate some of the speech processing disadvantages associated with hearing loss. However, little is known about the relation between hearing loss, hearing aid use, and their effects on higher-level language processes. This neuroimaging (fMRI) study examined these factors by measuring the comprehension and neural processing of simple and complex spoken sentences in hard-of-hearing older adults (n = 39). Neither hearing loss severity nor hearing aid experience influenced sentence comprehension at the behavioral level. In contrast, hearing loss severity was associated with increased activity in left superior frontal areas and the left anterior insula, but only when processing specific complex sentences (i.e. object-before-subject) compared to simple sentences. Longer hearing aid experience in a sub-set of participants (n = 19) was associated with recruitment of several areas outside of the core speech processing network in the right hemisphere, including the cerebellum, the precentral gyrus, and the cingulate cortex, but only when processing complex sentences. Overall, these results indicate that brain activation for language processing is affected by hearing loss as well as subsequent hearing aid use. Crucially, they show that these effects become apparent through investigation of complex but not simple sentences.

When Hearing Does Not Mean Understanding: On the Neural Processing of Syntactically Complex Sentences by Listeners With Hearing Loss.

Purpose Adults with mild-to-moderate age-related hearing loss typically exhibit issues with speech understanding, but their processing of syntactically complex sentences is not well understood. We test the hypothesis that listeners with hearing loss' difficulties with comprehension and processing of syntactically complex sentences are due to the processing of degraded input interfering with the successful processing of complex sentences. Method We performed a neuroimaging study with a sentence comprehension task, varying sentence complexity (through subject-object order and verb-arguments order) and cognitive demands (presence or absence of a secondary task) within subjects. Groups of older subjects with hearing loss (n = 20) and age-matched normal-hearing controls (n = 20) were tested. Results The comprehension data show effects of syntactic complexity and hearing ability, with normal-hearing controls outperforming listeners with hearing loss, seemingly more so on syntactically complex sentences. The secondary task did not influence off-line comprehension. The imaging data show effects of group, sentence complexity, and task, with listeners with hearing loss showing decreased activation in typical speech processing areas, such as the inferior frontal gyrus and superior temporal gyrus. No interactions between group, sentence complexity, and task were found in the neuroimaging data. Conclusions The results suggest that listeners with hearing loss process speech differently from their normal-hearing peers, possibly due to the increased demands of processing degraded auditory input. Increased cognitive demands by means of a secondary visual shape processing task influence neural sentence processing, but no evidence was found that it does so in a different way for listeners with hearing loss and normal-hearing listeners.

Brain Oscillation Entrainment by Perceptible and Non-perceptible Rhythmic Light Stimulation.

Objective and Background: Decades of research in the field of steady-state visual evoked potentials (SSVEPs) have revealed great potential of rhythmic light stimulation for brain-computer interfaces. Additionally, rhythmic light stimulation provides a non-invasive method for entrainment of oscillatory activity in the brain. Especially effective protocols enabling non-perceptible rhythmic stimulation and, thereby, reducing eye fatigue and user discomfort are favorable. Here, we investigate effects of (1) perceptible and (2) non-perceptible rhythmic light stimulation as well as attention-based effects of the stimulation by asking participants to focus (a) on the stimulation source directly in an overt attention condition or (b) on a cross-hair below the stimulation source in a covert attention condition. Method: SSVEPs at 10 Hz were evoked with a light-emitting diode (LED) driven by frequency-modulated signals and amplitudes of the current intensity either below or above a previously estimated individual threshold. Furthermore, we explored the effect of attention by asking participants to fixate on the LED directly in the overt attention condition and indirectly attend it in the covert attention condition. By measuring electroencephalography, we analyzed differences between conditions regarding the detection of reliable SSVEPs via the signal-to-noise ratio (SNR) and functional connectivity in occipito-frontal(-central) regions. Results: We could observe SSVEPs at 10 Hz for the perceptible and non-perceptible rhythmic light stimulation not only in the overt but also in the covert attention condition. The SNR and SSVEP amplitudes did not differ between the conditions and SNR values were in all except one participant above significance thresholds suggested by previous literature indicating reliable SSVEP responses. No difference between the conditions could be observed in the functional connectivity in occipito-frontal(-central) regions. Conclusion: The finding of robust SSVEPs even for non-intrusive rhythmic stimulation protocols below an individual perceptibility threshold and without direct fixation on the stimulation source reveals strong potential as a safe stimulation method for oscillatory entrainment in naturalistic applications.

Electrocorticography reveals continuous auditory and visual speech tracking in temporal and occipital cortex.

During natural speech perception, humans must parse temporally continuous auditory and visual speech signals into sequences of words. However, most studies of speech perception present only single words or syllables. We used electrocorticography (subdural electrodes implanted on the brains of epileptic patients) to investigate the neural mechanisms for processing continuous audiovisual speech signals consisting of individual sentences. Using partial correlation analysis, we found that posterior superior temporal gyrus (pSTG) and medial occipital cortex tracked both the auditory and the visual speech envelopes. These same regions, as well as inferior temporal cortex, responded more strongly to a dynamic video of a talking face compared to auditory speech paired with a static face. Occipital cortex and pSTG carry temporal information about both auditory and visual speech dynamics. Visual speech tracking in pSTG may be a mechanism for enhancing perception of degraded auditory speech.