The Spatiotemporal Neural Dynamics of Intersensory Attention Capture of Salient Stimuli: A Large-Scale Auditory-Visual Modeling Study.

The spatiotemporal dynamics of the neural mechanisms underlying endogenous (top-down) and exogenous (bottom-up) attention, and how attention is controlled or allocated in intersensory perception are not fully understood. We investigated these issues using a biologically realistic large-scale neural network model of visual-auditory object processing of short-term memory. We modeled and incorporated into our visual-auditory object-processing model the temporally changing neuronal mechanisms for the control of endogenous and exogenous attention. The model successfully performed various bimodal working memory tasks, and produced simulated behavioral and neural results that are consistent with experimental findings. Simulated fMRI data were generated that constitute predictions that human experiments could test. Furthermore, in our visual-auditory bimodality simulations, we found that increased working memory load in one modality would reduce the distraction from the other modality, and a possible network mediating this effect is proposed based on our model.

Quantifying Differences Between Passive and Task-Evoked Intrinsic Functional Connectivity in a Large-Scale Brain Simulation.

Establishing a connection between intrinsic and task-evoked brain activities is critical because it would provide a way to map task-related brain regions in patients unable to comply with such tasks. A crucial question within this realm is to what extent the execution of a cognitive task affects the intrinsic activity of brain regions not involved in the task. Computational models can be useful to answer this question because they allow us to distinguish task from nontask neural elements while giving us the effects of task execution on nontask regions of interest at the neuroimaging level. The quantification of those effects in a computational model would represent a step toward elucidating the intrinsic versus task-evoked connection. In this study we used computational modeling and graph theoretical metrics to quantify changes in intrinsic functional brain connectivity due to task execution. We used our large-scale neural modeling framework to embed a computational model of visual short-term memory into an empirically derived connectome. We simulated a neuroimaging study consisting of 10 subjects performing passive fixation (PF), passive viewing (PV), and delayed match-to-sample (DMS) tasks. We used the simulated blood oxygen level-dependent functional magnetic resonance imaging time series to calculate functional connectivity (FC) matrices and used those matrices to compute several graph theoretical measures. After determining that the simulated graph theoretical measures were largely consistent with experiments, we were able to quantify the differences between the graph metrics of the PF condition and those of the PV and DMS conditions. Thus, we show that we can use graph theoretical methods applied to simulated brain networks to aid in the quantification of changes in intrinsic brain FC during task execution. Our results represent a step toward establishing a connection between intrinsic and task-related brain activities.

Chronometry on Spike-LFP Responses Reveals the Functional Neural Circuitry of Early Auditory Cortex Underlying Sound Processing and Discrimination.

Animals and humans rapidly detect specific features of sounds, but the time courses of the underlying neural response for different stimulus categories is largely unknown. Furthermore, the intricate functional organization of auditory information processing pathways is poorly understood. Here, we computed neuronal response latencies from simultaneously recorded spike trains and local field potentials (LFPs) along the first two stages of cortical sound processing, primary auditory cortex (A1) and lateral belt (LB), of awake, behaving macaques. Two types of response latencies were measured for spike trains as well as LFPs: (1) onset latency, time-locked to onset of external auditory stimuli; and (2) selection latency, time taken from stimulus onset to a selective response to a specific stimulus category. Trial-by-trial LFP onset latencies predominantly reflecting synaptic input arrival typically preceded spike onset latencies, assumed to be representative of neuronal output indicating that both areas may receive input environmental signals and relay the information to the next stage. In A1, simple sounds, such as pure tones (PTs), yielded shorter spike onset latencies compared to complex sounds, such as monkey vocalizations ("Coos"). This trend was reversed in LB, indicating a hierarchical functional organization of auditory cortex in the macaque. LFP selection latencies in A1 were always shorter than those in LB for both PT and Coo reflecting the serial arrival of stimulus-specific information in these areas. Thus, chronometry on spike-LFP signals revealed some of the effective neural circuitry underlying complex sound discrimination.

Simulating laminar neuroimaging data for a visual delayed match-to-sample task.

Invasive electrophysiological and neuroanatomical studies in nonhuman mammalian experimental preparations have helped elucidate the lamina (layer) dependence of neural computations and interregional connections. Noninvasive functional neuroimaging can, in principle, resolve cortical laminae (layers), and thus provide insight into human neural computations and interregional connections. However human neuroimaging data are noisy and difficult to interpret; biologically realistic simulations can aid experimental interpretation by relating the neuroimaging data to simulated neural activity. We illustrate the potential of laminar neuroimaging by upgrading an existing large-scale, multiregion neural model that simulates a visual delayed match-to-sample (DMS) task. The new laminar-based neural unit incorporates spiny stellate, pyramidal, and inhibitory neural populations which are divided among supragranular, granular, and infragranular laminae (layers). We simulated neural activity which is translated into local field potential-like data used to simulate conventional and laminar fMRI activity. We implemented the laminar connectivity schemes proposed by Felleman and Van Essen (Cerebral Cortex, 1991) for interregional connections. The hemodynamic model that we employ is a modified version of one due to Heinzle et al. (Neuroimage, 2016) that incorporates the effects of draining veins. We show that the laminar version of the model replicates the findings of the existing model. The laminar model shows the finer structure in fMRI activity and functional connectivity. Laminar differences in the magnitude of neural activities are a prominent finding; these are also visible in the simulated fMRI. We illustrate differences between task and control conditions in the fMRI signal, and demonstrate differences in interregional laminar functional connectivity that reflect the underlying connectivity scheme. These results indicate that multi-layer computational models can aid in interpreting layer-specific fMRI, and suggest that increased use of laminar fMRI could provide unique and fundamental insights to human neuroscience.

Using a Large-scale Neural Model of Cortical Object Processing to Investigate the Neural Substrate for Managing Multiple Items in Short-term Memory.

Many cognitive and computational models have been proposed to help understand working memory. In this article, we present a simulation study of cortical processing of visual objects during several working memory tasks using an extended version of a previously constructed large-scale neural model [Tagamets, M. A., & Horwitz, B. Integrating electrophysiological and anatomical experimental data to create a large-scale model that simulates a delayed match-to-sample human brain imaging study. Cerebral Cortex, 8, 310-320, 1998]. The original model consisted of arrays of Wilson-Cowan type of neuronal populations representing primary and secondary visual cortices, inferotemporal (IT) cortex, and pFC. We added a module representing entorhinal cortex, which functions as a gating module. We successfully implemented multiple working memory tasks using the same model and produced neuronal patterns in visual cortex, IT cortex, and pFC that match experimental findings. These working memory tasks can include distractor stimuli or can require that multiple items be retained in mind during a delay period (Sternberg's task). Besides electrophysiology data and behavioral data, we also generated fMRI BOLD time series from our simulation. Our results support the involvement of IT cortex in working memory maintenance and suggest the cortical architecture underlying the neural mechanisms mediating particular working memory tasks. Furthermore, we noticed that, during simulations of memorizing a list of objects, the first and last items in the sequence were recalled best, which may implicate the neural mechanism behind this important psychological effect (i.e., the primacy and recency effect).

The Functional Overlap of Executive Control and Language Processing in Bilinguals.

The need to control multiple languages is thought to require domain-general executive control (EC) in bilinguals such that the EC and language systems become interdependent. However, there has been no systematic investigation into how and where EC and language processes overlap in the bilingual brain. If the concurrent recruitment of EC during bilingual language processing is domain-general and extends to non-linguistic EC, we hypothesize that regions commonly involvement in language processing, linguistic EC, and non-linguistic EC may be selectively altered in bilinguals compared to monolinguals. A conjunction of functional magnetic resonance imaging (fMRI) data from a flanker task with linguistic and nonlinguistic distractors and a semantic categorization task showed functional overlap in the left inferior frontal gyrus (LIFG) in bilinguals, whereas no overlap occurred in monolinguals. This research therefore identifies a neural locus of functional overlap of language and EC in the bilingual brain.

Embedding Task-Based Neural Models into a Connectome-Based Model of the Cerebral Cortex.

A number of recent efforts have used large-scale, biologically realistic, neural models to help understand the neural basis for the patterns of activity observed in both resting state and task-related functional neural imaging data. An example of the former is The Virtual Brain (TVB) software platform, which allows one to apply large-scale neural modeling in a whole brain framework. TVB provides a set of structural connectomes of the human cerebral cortex, a collection of neural processing units for each connectome node, and various forward models that can convert simulated neural activity into a variety of functional brain imaging signals. In this paper, we demonstrate how to embed a previously or newly constructed task-based large-scale neural model into the TVB platform. We tested our method on a previously constructed large-scale neural model (LSNM) of visual object processing that consisted of interconnected neural populations that represent, primary and secondary visual, inferotemporal, and prefrontal cortex. Some neural elements in the original model were "non-task-specific" (NS) neurons that served as noise generators to "task-specific" neurons that processed shapes during a delayed match-to-sample (DMS) task. We replaced the NS neurons with an anatomical TVB connectome model of the cerebral cortex comprising 998 regions of interest interconnected by white matter fiber tract weights. We embedded our LSNM of visual object processing into corresponding nodes within the TVB connectome. Reciprocal connections between TVB nodes and our task-based modules were included in this framework. We ran visual object processing simulations and showed that the TVB simulator successfully replaced the noise generation originally provided by NS neurons; i.e., the DMS tasks performed with the hybrid LSNM/TVB simulator generated equivalent neural and fMRI activity to that of the original task-based models. Additionally, we found partial agreement between the functional connectivities using the hybrid LSNM/TVB model and the original LSNM. Our framework thus presents a way to embed task-based neural models into the TVB platform, enabling a better comparison between empirical and computational data, which in turn can lead to a better understanding of how interacting neural populations give rise to human cognitive behaviors.

PreSMA stimulation changes task-free functional connectivity in the fronto-basal-ganglia that correlates with response inhibition efficiency.

Previous work using transcranial magnetic stimulation (TMS) demonstrated that the right presupplementary motor area (preSMA), a node in the fronto-basal-ganglia network, is critical for response inhibition. However, TMS influences interconnected regions, raising the possibility of a link between the preSMA activity and the functional connectivity within the network. To understand this relationship, we applied single-pulse TMS to the right preSMA during functional magnetic resonance imaging when the subjects were at rest to examine changes in neural activity and functional connectivity within the network in relation to the efficiency of response inhibition evaluated with a stop-signal task. The results showed that preSMA-TMS increased activation in the right inferior-frontal cortex (rIFC) and basal ganglia and modulated their task-free functional connectivity. Both the TMS-induced changes in the basal-ganglia activation and the functional connectivity between rIFC and left striatum, and of the overall network correlated with the efficiency of response inhibition and with the white-matter microstructure along the preSMA-rIFC pathway. These results suggest that the task-free functional and structural connectivity between the rIFCop and basal ganglia are critical to the efficiency of response inhibition. Hum Brain Mapp 37:3236-3249, 2016. © 2016 Wiley Periodicals, Inc.

The Functional Connectome of Speech Control.

In the past few years, several studies have been directed to understanding the complexity of functional interactions between different brain regions during various human behaviors. Among these, neuroimaging research installed the notion that speech and language require an orchestration of brain regions for comprehension, planning, and integration of a heard sound with a spoken word. However, these studies have been largely limited to mapping the neural correlates of separate speech elements and examining distinct cortical or subcortical circuits involved in different aspects of speech control. As a result, the complexity of the brain network machinery controlling speech and language remained largely unknown. Using graph theoretical analysis of functional MRI (fMRI) data in healthy subjects, we quantified the large-scale speech network topology by constructing functional brain networks of increasing hierarchy from the resting state to motor output of meaningless syllables to complex production of real-life speech as well as compared to non-speech-related sequential finger tapping and pure tone discrimination networks. We identified a segregated network of highly connected local neural communities (hubs) in the primary sensorimotor and parietal regions, which formed a commonly shared core hub network across the examined conditions, with the left area 4p playing an important role in speech network organization. These sensorimotor core hubs exhibited features of flexible hubs based on their participation in several functional domains across different networks and ability to adaptively switch long-range functional connectivity depending on task content, resulting in a distinct community structure of each examined network. Specifically, compared to other tasks, speech production was characterized by the formation of six distinct neural communities with specialized recruitment of the prefrontal cortex, insula, putamen, and thalamus, which collectively forged the formation of the functional speech connectome. In addition, the observed capacity of the primary sensorimotor cortex to exhibit operational heterogeneity challenged the established concept of unimodality of this region.

Detecting Functional Connectivity During Audiovisual Integration with MEG: A Comparison of Connectivity Metrics.

In typical magnetoencephalography and/or electroencephalography functional connectivity analysis, researchers select one of several methods that measure a relationship between regions to determine connectivity, such as coherence, power correlations, and others. However, it is largely unknown if some are more suited than others for various types of investigations. In this study, the authors investigate seven connectivity metrics to evaluate which, if any, are sensitive to audiovisual integration by contrasting connectivity when tracking an audiovisual object versus connectivity when tracking a visual object uncorrelated with the auditory stimulus. The authors are able to assess the metrics' performances at detecting audiovisual integration by investigating connectivity between auditory and visual areas. Critically, the authors perform their investigation on a whole-cortex all-to-all mapping, avoiding confounds introduced in seed selection. The authors find that amplitude-based connectivity measures in the beta band detect strong connections between visual and auditory areas during audiovisual integration, specifically between V4/V5 and auditory cortices in the right hemisphere. Conversely, phase-based connectivity measures in the beta band as well as phase and power measures in alpha, gamma, and theta do not show connectivity between audiovisual areas. The authors postulate that while beta power correlations detect audiovisual integration in the current experimental context, it may not always be the best measure to detect connectivity. Instead, it is likely that the brain utilizes a variety of mechanisms in neuronal communication that may produce differential types of temporal relationships.

Processing of harmonics in the lateral belt of macaque auditory cortex.

Many speech sounds and animal vocalizations contain components, referred to as complex tones, that consist of a fundamental frequency (F0) and higher harmonics. In this study we examined single-unit activity recorded in the core (A1) and lateral belt (LB) areas of auditory cortex in two rhesus monkeys as they listened to pure tones and pitch-shifted conspecific vocalizations ("coos"). The latter consisted of complex-tone segments in which F0 was matched to a corresponding pure-tone stimulus. In both animals, neuronal latencies to pure-tone stimuli at the best frequency (BF) were ~10 to 15 ms longer in LB than in A1. This might be expected, since LB is considered to be at a hierarchically higher level than A1. On the other hand, the latency of LB responses to coos was ~10 to 20 ms shorter than to the corresponding pure-tone BF, suggesting facilitation in LB by the harmonics. This latency reduction by coos was not observed in A1, resulting in similar coo latencies in A1 and LB. Multi-peaked neurons were present in both A1 and LB; however, harmonically-related peaks were observed in LB for both early and late response components, whereas in A1 they were observed only for late components. Our results suggest that harmonic features, such as relationships between specific frequency intervals of communication calls, are processed at relatively early stages of the auditory cortical pathway, but preferentially in LB.

Interpreting the effects of altered brain anatomical connectivity on fMRI functional connectivity: a role for computational neural modeling.

Recently, there have been a large number of studies using resting state fMRI to characterize abnormal brain connectivity in patients with a variety of neurological, psychiatric, and developmental disorders. However, interpreting what the differences in resting state fMRI functional connectivity (rsfMRI-FC) actually reflect in terms of the underlying neural pathology has proved to be elusive because of the complexity of brain anatomical connectivity. The same is the case for task-based fMRI studies. In the last few years, several groups have used large-scale neural modeling to help provide some insight into the relationship between brain anatomical connectivity and the corresponding patterns of fMRI-FC. In this paper we review several efforts at using large-scale neural modeling to investigate the relationship between structural connectivity and functional/effective connectivity to determine how alterations in structural connectivity are manifested in altered patterns of functional/effective connectivity. Because the alterations made in the anatomical connectivity between specific brain regions in the model are known in detail, one can use the results of these simulations to determine the corresponding alterations in rsfMRI-FC. Many of these simulation studies found that structural connectivity changes do not necessarily result in matching changes in functional/effective connectivity in the areas of structural modification. Often, it was observed that increases in functional/effective connectivity in the altered brain did not necessarily correspond to increases in the strength of the anatomical connection weights. Note that increases in rsfMRI-FC in patients have been interpreted in some cases as resulting from neural plasticity. These results suggest that this interpretation can be mistaken. The relevance of these simulation findings to the use of functional/effective fMRI connectivity as biomarkers for brain disorders is also discussed.

Separating lexical-semantic access from other mnemonic processes in picture-name verification.

We present a novel paradigm to identify shared and unique brain regions underlying non-semantic, non-phonological, abstract, audio-visual (AV) memory vs. naming using a longitudinal functional magnetic resonance imaging experiment. Participants were trained to associate novel AV stimulus pairs containing hidden linguistic content. Half of the stimulus pairs were distorted images of animals and sine-wave speech versions of the animal's name. Images and sounds were distorted in such a way as to make their linguistic content easily recognizable only after being made aware of its existence. Memory for the pairings was tested by presenting an AV pair and asking participants to verify if the two stimuli formed a learned pairing. After memory testing, the hidden linguistic content was revealed and participants were tested again on their recollection of the pairings in this linguistically informed state. Once informed, the AV verification task could be performed by naming the picture. There was substantial overlap between the regions involved in recognition of non-linguistic sensory memory and naming, suggesting a strong relation between them. Contrasts between sessions identified left angular gyrus and middle temporal gyrus as key additional players in the naming network. Left inferior frontal regions participated in both naming and non-linguistic AV memory suggesting the region is responsible for AV memory independent of phonological content contrary to previous proposals. Functional connectivity between angular gyrus and left inferior frontal gyrus and left middle temporal gyrus increased when performing the AV task as naming. The results are consistent with the hypothesis that, at the spatial resolution of fMRI, the regions that facilitate non-linguistic AV associations are a subset of those that facilitate naming though reorganized into distinct networks.

Identifying effective connectivity parameters in simulated fMRI: a direct comparison of switching linear dynamic system, stochastic dynamic causal, and multivariate autoregressive models.

The number and variety of connectivity estimation methods is likely to continue to grow over the coming decade. Comparisons between methods are necessary to prune this growth to only the most accurate and robust methods. However, the nature of connectivity is elusive with different methods potentially attempting to identify different aspects of connectivity. Commonalities of connectivity definitions across methods upon which base direct comparisons can be difficult to derive. Here, we explicitly define "effective connectivity" using a common set of observation and state equations that are appropriate for three connectivity methods: dynamic causal modeling (DCM), multivariate autoregressive modeling (MAR), and switching linear dynamic systems for fMRI (sLDSf). In addition while deriving this set, we show how many other popular functional and effective connectivity methods are actually simplifications of these equations. We discuss implications of these connections for the practice of using one method to simulate data for another method. After mathematically connecting the three effective connectivity methods, simulated fMRI data with varying numbers of regions and task conditions is generated from the common equation. This simulated data explicitly contains the type of the connectivity that the three models were intended to identify. Each method is applied to the simulated data sets and the accuracy of parameter identification is analyzed. All methods perform above chance levels at identifying correct connectivity parameters. The sLDSf method was superior in parameter estimation accuracy to both DCM and MAR for all types of comparisons.

Assessing crossmodal matching of abstract auditory and visual stimuli in posterior superior temporal sulcus with MEG.

Associating crossmodal auditory and visual stimuli is an important component of perception, with the posterior superior temporal sulcus (pSTS) hypothesized to support this. However, recent evidence has argued that the pSTS serves to associate two stimuli irrespective of modality. To examine the contribution of pSTS to crossmodal recognition, participants (N=13) learned 12 abstract, non-linguistic pairs of stimuli over 3weeks. These paired associates comprised four types: auditory-visual (AV), auditory-auditory (AA), visual-auditory (VA), and visual-visual (VV). At week four, participants were scanned using magnetoencephalography (MEG) while performing a correct/incorrect judgment on pairs of items. Using an implementation of synthetic aperture magnetometry that computes real statistics across trials (SAMspm), we directly contrasted crossmodal (AV and VA) with unimodal (AA and VV) pairs from stimulus-onset to 2s in theta (4-8Hz), alpha (9-15Hz), beta (16-30Hz), and gamma (31-50Hz) frequencies. We found pSTS showed greater desynchronization in the beta frequency for crossmodal compared with unimodal trials, suggesting greater activity during the crossmodal pairs, which was not influenced by congruency of the paired stimuli. Using a sliding window SAM analysis, we found the timing of this difference began in a window from 250 to 750ms after stimulus-onset. Further, when we directly contrasted all sub-types of paired associates from stimulus-onset to 2s, we found that pSTS seemed to respond to dynamic, auditory stimuli, rather than crossmodal stimuli per se. These findings support an early role for pSTS in the processing of dynamic, auditory stimuli, and do not support claims that pSTS is responsible for associating two stimuli irrespective of their modality.

Can quantum probability help analyze the behavior of functional brain networks?

Pothos & Busemeyer (P&B) argue how key concepts of quantum probability, for example, order/context, interference, superposition, and entanglement, can be used in cognitive modeling. Here, we suggest that these concepts can be extended to analyze neurophysiological measurements of cognitive tasks in humans, especially in functional neuroimaging investigations of large-scale brain networks.

Speech-induced striatal dopamine release is left lateralized and coupled to functional striatal circuits in healthy humans: a combined PET, fMRI and DTI study.

Considerable progress has been recently made in understanding the brain mechanisms underlying speech and language control. However, the neurochemical underpinnings of normal speech production remain largely unknown. We investigated the extent of striatal endogenous dopamine release and its influences on the organization of functional striatal speech networks during production of meaningful English sentences using a combination of positron emission tomography (PET) with the dopamine D(2)/D(3) receptor radioligand [(11)C]raclopride and functional MRI (fMRI). In addition, we used diffusion tensor tractography (DTI) to examine the extent of dopaminergic modulatory influences on striatal structural network organization. We found that, during sentence production, endogenous dopamine was released in the ventromedial portion of the dorsal striatum, in both its associative and sensorimotor functional divisions. In the associative striatum, speech-induced dopamine release established a significant relationship with neural activity and influenced the left-hemispheric lateralization of striatal functional networks. In contrast, there were no significant effects of endogenous dopamine release on the lateralization of striatal structural networks. Our data provide the first evidence for endogenous dopamine release in the dorsal striatum during normal speaking and point to the possible mechanisms behind the modulatory influences of dopamine on the organization of functional brain circuits controlling normal human speech.