Identifying Naturalistic Movies from Human Brain Activity with High-Density Diffuse Optical Tomography.

Modern neuroimaging modalities, particularly functional MRI (fMRI), can decode detailed human experiences. Thousands of viewed images can be identified or classified, and sentences can be reconstructed. Decoding paradigms often leverage encoding models that reduce the stimulus space into a smaller yet generalizable feature set. However, the neuroimaging devices used for detailed decoding are non-portable, like fMRI, or invasive, like electrocorticography, excluding application in naturalistic use. Wearable, non-invasive, but lower-resolution devices such as electroencephalography and functional near-infrared spectroscopy (fNIRS) have been limited to decoding between stimuli used during training. Herein we develop and evaluate model-based decoding with high-density diffuse optical tomography (HD-DOT), a higher-resolution expansion of fNIRS with demonstrated promise as a surrogate for fMRI. Using a motion energy model of visual content, we decoded the identities of novel movie clips outside the training set with accuracy far above chance for single-trial decoding. Decoding was robust to modulations of testing time window, different training and test imaging sessions, hemodynamic contrast, and optode array density. Our results suggest that HD-DOT can translate detailed decoding into naturalistic use.

Ultra-high density imaging arrays for diffuse optical tomography of human brain improve resolution, signal-to-noise, and information decoding.

Functional magnetic resonance imaging (fMRI) has dramatically advanced non-invasive human brain mapping and decoding. Functional near-infrared spectroscopy (fNIRS) and high-density diffuse optical tomography (HD-DOT) non-invasively measure blood oxygen fluctuations related to brain activity, like fMRI, at the brain surface, using more-lightweight equipment that circumvents ergonomic and logistical limitations of fMRI. HD-DOT grids have smaller inter-optode spacing (∼13 mm) than sparse fNIRS (∼30 mm) and therefore provide higher image quality, with spatial resolution ∼1/2 that of fMRI. Herein, simulations indicated reducing inter-optode spacing to 6.5 mm would further improve image quality and noise-resolution tradeoff, with diminishing returns below 6.5 mm. We then constructed an ultra-high-density DOT system (6.5-mm spacing) with 140 dB dynamic range that imaged stimulus-evoked activations with 30-50% higher spatial resolution and repeatable multi-focal activity with excellent agreement with participant-matched fMRI. Further, this system decoded visual stimulus position with 19-35% lower error than previous HD-DOT, throughout occipital cortex.

Scalable, modular continuous wave functional near-infrared spectroscopy system (Spotlight).

Significance: We present a fiberless, portable, and modular continuous wave-functional near-infrared spectroscopy system, Spotlight, consisting of multiple palm-sized modules-each containing high-density light-emitting diode and silicon photomultiplier detector arrays embedded in a flexible membrane that facilitates optode coupling to scalp curvature. Aim: Spotlight's goal is to be a more portable, accessible, and powerful functional near-infrared spectroscopy (fNIRS) device for neuroscience and brain-computer interface (BCI) applications. We hope that the Spotlight designs we share here can spur more advances in fNIRS technology and better enable future non-invasive neuroscience and BCI research. Approach: We report sensor characteristics in system validation on phantoms and motor cortical hemodynamic responses in a human finger-tapping experiment, where subjects wore custom 3D-printed caps with two sensor modules. Results: The task conditions can be decoded offline with a median accuracy of 69.6%, reaching 94.7% for the best subject, and at a comparable accuracy in real time for a subset of subjects. We quantified how well the custom caps fitted to each subject and observed that better fit leads to more observed task-dependent hemodynamic response and better decoding accuracy. Conclusions: The advances presented here should serve to make fNIRS more accessible for BCI applications.

High-sensitivity multispeckle diffuse correlation spectroscopy.

Significance: Cerebral blood flow is an important biomarker of brain health and function as it regulates the delivery of oxygen and substrates to tissue and the removal of metabolic waste products. Moreover, blood flow changes in specific areas of the brain are correlated with neuronal activity in those areas. Diffuse correlation spectroscopy (DCS) is a promising noninvasive optical technique for monitoring cerebral blood flow and for measuring cortex functional activation tasks. However, the current state-of-the-art DCS adoption is hindered by a trade-off between sensitivity to the cortex and signal-to-noise ratio (SNR). Aim: We aim to develop a scalable method that increases the sensitivity of DCS instruments. Approach: We report on a multispeckle DCS (mDCS) approach that is based on a 1024-pixel single-photon avalanche diode (SPAD) camera. Our approach is scalable to > 100,000 independent speckle measurements since large-pixel-count SPAD cameras are becoming available, owing to the investments in LiDAR technology for automotive and augmented reality applications. Results: We demonstrated a 32-fold increase in SNR with respect to traditional single-speckle DCS. Conclusion: A mDCS system that is based on a SPAD camera serves as a scalable method toward high-sensitivity DCS measurements, thus enabling both high sensitivity to the cortex and high SNR.

Neural Reconstruction Integrity: A Metric for Assessing the Connectivity Accuracy of Reconstructed Neural Networks.

Neuroscientists are actively pursuing high-precision maps, or graphs consisting of networks of neurons and connecting synapses in mammalian and non-mammalian brains. Such graphs, when coupled with physiological and behavioral data, are likely to facilitate greater understanding of how circuits in these networks give rise to complex information processing capabilities. Given that the automated or semi-automated methods required to achieve the acquisition of these graphs are still evolving, we developed a metric for measuring the performance of such methods by comparing their output with those generated by human annotators ("ground truth" data). Whereas classic metrics for comparing annotated neural tissue reconstructions generally do so at the voxel level, the metric proposed here measures the "integrity" of neurons based on the degree to which a collection of synaptic terminals belonging to a single neuron of the reconstruction can be matched to those of a single neuron in the ground truth data. The metric is largely insensitive to small errors in segmentation and more directly measures accuracy of the generated brain graph. It is our hope that use of the metric will facilitate the broader community's efforts to improve upon existing methods for acquiring brain graphs. Herein we describe the metric in detail, provide demonstrative examples of the intuitive scores it generates, and apply it to a synthesized neural network with simulated reconstruction errors. Demonstration code is available.

Training Humans to Categorize Monkey Calls: Auditory Feature- and Category-Selective Neural Tuning Changes.

Grouping auditory stimuli into common categories is essential for a variety of auditory tasks, including speech recognition. We trained human participants to categorize auditory stimuli from a large novel set of morphed monkey vocalizations. Using fMRI-rapid adaptation (fMRI-RA) and multi-voxel pattern analysis (MVPA) techniques, we gained evidence that categorization training results in two distinct sets of changes: sharpened tuning to monkey call features (without explicit category representation) in left auditory cortex and category selectivity for different types of calls in lateral prefrontal cortex. In addition, the sharpness of neural selectivity in left auditory cortex, as estimated with both fMRI-RA and MVPA, predicted the steepness of the categorical boundary, whereas categorical judgment correlated with release from adaptation in the left inferior frontal gyrus. These results support the theory that auditory category learning follows a two-stage model analogous to the visual domain, suggesting general principles of perceptual category learning in the human brain.

Semantic attributes are encoded in human electrocorticographic signals during visual object recognition.

Non-invasive neuroimaging studies have shown that semantic category and attribute information are encoded in neural population activity. Electrocorticography (ECoG) offers several advantages over non-invasive approaches, but the degree to which semantic attribute information is encoded in ECoG responses is not known. We recorded ECoG while patients named objects from 12 semantic categories and then trained high-dimensional encoding models to map semantic attributes to spectral-temporal features of the task-related neural responses. Using these semantic attribute encoding models, untrained objects were decoded with accuracies comparable to whole-brain functional Magnetic Resonance Imaging (fMRI), and we observed that high-gamma activity (70-110Hz) at basal occipitotemporal electrodes was associated with specific semantic dimensions (manmade-animate, canonically large-small, and places-tools). Individual patient results were in close agreement with reports from other imaging modalities on the time course and functional organization of semantic processing along the ventral visual pathway during object recognition. The semantic attribute encoding model approach is critical for decoding objects absent from a training set, as well as for studying complex semantic encodings without artificially restricting stimuli to a small number of semantic categories.

Classification of pilot-induced oscillations during in-flight piloting exercises using dry EEG sensor recordings.

Pilot-Induced Oscillations (PIOs) are potentially hazardous piloting phenomena in which a pilot's control-inputs and the aircraft control-responses have (for any of a number of possible reasons) become out of phase. During PIOs, aggressive over-controlling on the part of the pilot in order to overcome a perceived lack of control can lead to complete loss of aircraft control. This study shows data recorded from a Cognionics dry electrode system during actual flight exercises can be used on a second-to-second basis to classify whether a pilot was undergoing a PIO event or if a PIO was imminent. If such PIO predictions could be made with adequate accuracy and robustness in real-time, they could form the basis of systems aimed at detecting and/or mitigating PIOs.

An automated images-to-graphs framework for high resolution connectomics.

Reconstructing a map of neuronal connectivity is a critical challenge in contemporary neuroscience. Recent advances in high-throughput serial section electron microscopy (EM) have produced massive 3D image volumes of nanoscale brain tissue for the first time. The resolution of EM allows for individual neurons and their synaptic connections to be directly observed. Recovering neuronal networks by manually tracing each neuronal process at this scale is unmanageable, and therefore researchers are developing automated image processing modules. Thus far, state-of-the-art algorithms focus only on the solution to a particular task (e.g., neuron segmentation or synapse identification). In this manuscript we present the first fully-automated images-to-graphs pipeline (i.e., a pipeline that begins with an imaged volume of neural tissue and produces a brain graph without any human interaction). To evaluate overall performance and select the best parameters and methods, we also develop a metric to assess the quality of the output graphs. We evaluate a set of algorithms and parameters, searching possible operating points to identify the best available brain graph for our assessment metric. Finally, we deploy a reference end-to-end version of the pipeline on a large, publicly available data set. This provides a baseline result and framework for community analysis and future algorithm development and testing. All code and data derivatives have been made publicly available in support of eventually unlocking new biofidelic computational primitives and understanding of neuropathologies.

Automatic phoneme category selectivity in the dorsal auditory stream.

Debates about motor theories of speech perception have recently been reignited by a burst of reports implicating premotor cortex (PMC) in speech perception. Often, however, these debates conflate perceptual and decision processes. Evidence that PMC activity correlates with task difficulty and subject performance suggests that PMC might be recruited, in certain cases, to facilitate category judgments about speech sounds (rather than speech perception, which involves decoding of sounds). However, it remains unclear whether PMC does, indeed, exhibit neural selectivity that is relevant for speech decisions. Further, it is unknown whether PMC activity in such cases reflects input via the dorsal or ventral auditory pathway, and whether PMC processing of speech is automatic or task-dependent. In a novel modified categorization paradigm, we presented human subjects with paired speech sounds from a phonetic continuum but diverted their attention from phoneme category using a challenging dichotic listening task. Using fMRI rapid adaptation to probe neural selectivity, we observed acoustic-phonetic selectivity in left anterior and left posterior auditory cortical regions. Conversely, we observed phoneme-category selectivity in left PMC that correlated with explicit phoneme-categorization performance measured after scanning, suggesting that PMC recruitment can account for performance on phoneme-categorization tasks. Structural equation modeling revealed connectivity from posterior, but not anterior, auditory cortex to PMC, suggesting a dorsal route for auditory input to PMC. Our results provide evidence for an account of speech processing in which the dorsal stream mediates automatic sensorimotor integration of speech and may be recruited to support speech decision tasks.

Functional correlates of the anterolateral processing hierarchy in human auditory cortex.

Converging evidence supports the hypothesis that an anterolateral processing pathway mediates sound identification in auditory cortex, analogous to the role of the ventral cortical pathway in visual object recognition. Studies in nonhuman primates have characterized the anterolateral auditory pathway as a processing hierarchy, composed of three anatomically and physiologically distinct initial stages: core, belt, and parabelt. In humans, potential homologs of these regions have been identified anatomically, but reliable and complete functional distinctions between them have yet to be established. Because the anatomical locations of these fields vary across subjects, investigations of potential homologs between monkeys and humans require these fields to be defined in single subjects. Using functional MRI, we presented three classes of sounds (tones, band-passed noise bursts, and conspecific vocalizations), equivalent to those used in previous monkey studies. In each individual subject, three regions showing functional similarities to macaque core, belt, and parabelt were readily identified. Furthermore, the relative sizes and locations of these regions were consistent with those reported in human anatomical studies. Our results demonstrate that the functional organization of the anterolateral processing pathway in humans is largely consistent with that of nonhuman primates. Because our scanning sessions last only 15 min/subject, they can be run in conjunction with other scans. This will enable future studies to characterize functional modules in human auditory cortex at a level of detail previously possible only in visual cortex. Furthermore, the approach of using identical schemes in both humans and monkeys will aid with establishing potential homologies between them.

Dysregulation of limbic and auditory networks in tinnitus.

Tinnitus is a common disorder characterized by ringing in the ear in the absence of sound. Converging evidence suggests that tinnitus pathophysiology involves damage to peripheral and/or central auditory pathways. However, whether auditory system dysfunction is sufficient to explain chronic tinnitus is unclear, especially in light of evidence implicating other networks, including the limbic system. Using functional magnetic resonance imaging and voxel-based morphometry, we assessed tinnitus-related functional and anatomical anomalies in auditory and limbic networks. Moderate hyperactivity was present in the primary and posterior auditory cortices of tinnitus patients. However, the nucleus accumbens exhibited the greatest degree of hyperactivity, specifically to sounds frequency-matched to patients' tinnitus. Complementary structural differences were identified in ventromedial prefrontal cortex, another limbic structure heavily connected to the nucleus accumbens. Furthermore, tinnitus-related anomalies were intercorrelated in the two limbic regions and between limbic and primary auditory areas, indicating the importance of auditory-limbic interactions in tinnitus.