Low-level language processing in brain-injured patients.

Assessing cognitive function-especially language processing-in severely brain-injured patients is critical for prognostication, care, and development of communication devices (e.g. brain-computer interfaces). In patients with diminished motor function, language processing has been probed using EEG measures of command-following in motor imagery tasks. While such tests eliminate the need for motor response, they require sustained attention. However, passive listening tasks, with an EEG response measure can reduce both motor and attentional demands. These considerations motivated the development of two assays of low-level language processing-identification of differential phoneme-class responses and tracking of the natural speech envelope. This cross-sectional study looks at a cohort of 26 severely brain-injured patient subjects and 10 healthy controls. Patients' level of function was assessed via the coma recovery scale-revised at the bedside. Patients were also tested for command-following via EEG and/or MRI assays of motor imagery. For the present investigation, EEG was recorded while presenting a 148 s audio clip of Alice in Wonderland. Time-locked EEG responses to phoneme classes were extracted and compared to determine a differential phoneme-class response. Tracking of the natural speech envelope was assessed from the same recordings by cross-correlating the EEG response with the speech envelope. In healthy controls, the dynamics of the two measures were temporally similar but spatially different: a central parieto-occipital component of differential phoneme-class response was absent in the natural speech envelope response. The differential phoneme-class response was present in all patient subjects, including the six classified as vegetative state/unresponsive wakefulness syndrome by behavioural assessment. However, patient subjects with evidence of language processing either by behavioural assessment or motor imagery tests had an early bilateral response in the first 50 ms that was lacking in patient subjects without any evidence of language processing. The natural speech envelope tracking response was also present in all patient subjects and responses in the first 100 ms distinguished patient subjects with evidence of language processing. Specifically, patient subjects with evidence of language processing had a more global response in the first 100 ms whereas those without evidence of language processing had a frontopolar response in that period. In summary, we developed two passive EEG-based methods to probe low-level language processing in severely brain-injured patients. In our cohort, both assays showed a difference between patient subjects with evidence of command-following and those with no evidence of command-following: a more prominent early bilateral response component.

Discrimination of textures with spatial correlations and multiple gray levels.

Analysis of visual texture is important for many key steps in early vision. We study visual sensitivity to image statistics in three families of textures that include multiple gray levels and correlations in two spatial dimensions. Sensitivities to positive and negative correlations are approximately independent of correlation sign, and signals from different kinds of correlations combine quadratically. We build a computational model, fully constrained by prior studies of sensitivity to uncorrelated textures and black-and-white textures with spatial correlations. The model accounts for many features of the new data, including sign-independence, quadratic combination, and the dependence on gray-level distribution.

Functional recursion of orientation cues in figure-ground separation.

Visual texture is an important cue to figure-ground organization. While processing of texture differences is a prerequisite for the use of this cue to extract figure-ground organization, these stages are distinct processes. One potential indicator of this distinction is the possibility that texture statistics play a different role in the figure vs. in the ground. To determine whether this is the case, we probed figure-ground processing with a family of local image statistics that specified textures that varied in the strength and spatial scale of structure, and the extent to which features are oriented. For image statistics that generated approximately isotropic textures, the threshold for identification of figure-ground structure was determined by the difference in correlation strength in figure vs. ground, independent of whether the correlations were present in figure, ground, or both. However, for image statistics with strong orientation content, thresholds were up to two times higher for correlations in the ground, vs. the figure. This held equally for texture-defined objects with convex or concave boundaries, indicating that these threshold differences are driven by border ownership, not boundary shape. Similar threshold differences were found for presentation times ranging from 125 to 500 ms. These findings identify a qualitative difference in how texture is used for figure-ground analysis, vs. texture discrimination. Additionally, it reveals a functional recursion: texture differences are needed to identify tentative boundaries and consequent scene organization into figure and ground, but then scene organization modifies sensitivity to texture differences according to the figure-ground assignment.

Electrophysiological correlates of thalamocortical function in acute severe traumatic brain injury.

Tools assaying the neural networks that modulate consciousness may facilitate tracking of recovery after acute severe brain injury. The ABCD framework classifies resting-state EEG into categories reflecting levels of thalamocortical network function that correlate with outcome in post-cardiac arrest coma. In this longitudinal cohort study, we applied the ABCD framework to 20 patients with acute severe traumatic brain injury requiring intensive care (12 of whom were also studied at ≥6-months post-injury) and 16 healthy controls. We tested four hypotheses: 1) EEG ABCD classifications are spatially heterogeneous and temporally variable; 2) ABCD classifications improve longitudinally, commensurate with the degree of behavioral recovery; 3) ABCD classifications correlate with behavioral level of consciousness; and 4) the Coma Recovery Scale-Revised arousal facilitation protocol yields improved ABCD classifications. Channel-level EEG power spectra were classified based on spectral peaks within pre-defined frequency bands: 'A' = no peaks above delta (<4 Hz) range (complete thalamocortical disruption); 'B' = theta (4-8 Hz) peak (severe thalamocortical disruption); 'C' = theta and beta (13-24 Hz) peaks (moderate thalamocortical disruption); or 'D' = alpha (8-13 Hz) and beta peaks (normal thalamocortical function). Acutely, 95% of patients demonstrated 'D' signals in at least one channel but exhibited within-session temporal variability and spatial heterogeneity in the proportion of different channel-level ABCD classifications. By contrast, healthy participants and patients at follow-up consistently demonstrated signals corresponding to intact thalamocortical network function. Patients demonstrated longitudinal improvement in ABCD classifications (p < .05) and ABCD classification distinguished patients with and without command-following in the subacute-to-chronic phase of recovery (p < .01). In patients studied acutely, ABCD classifications improved after the Coma Recovery Scale-Revised arousal facilitation protocol (p < .05) but did not correspond with behavioral level of consciousness. These findings support the use of the ABCD framework to characterize channel-level EEG dynamics and track fluctuations in functional thalamocortical network integrity in spatial detail.

Visual Search for Circumscribed Interests in Autism Is Similar to That of Neurotypical Individuals.

Intense interests are a core symptom of autism spectrum disorders (ASD) and can be all-encompassing for affected individuals. This observation raises the hypothesis that intense interests in ASD are related to pervasive changes in visual processing for objects within that category, including visual search. We assayed visual processing with two novel tasks, targeting category search and exemplar search. For each task, three kinds of stimuli were used: faces, houses, and images personalized to each participant's interest. 25 children and adults with ASD were compared to 25 neurotypical (NT) children and adults. We found no differences in either visual search task between ASD and NT controls for interests. Thus, pervasive alterations in perception are not likely to account for ASD behavioral symptoms.

Efficient coding of natural scene statistics predicts discrimination thresholds for grayscale textures.

Previously, in Hermundstad et al., 2014, we showed that when sampling is limiting, the efficient coding principle leads to a 'variance is salience' hypothesis, and that this hypothesis accounts for visual sensitivity to binary image statistics. Here, using extensive new psychophysical data and image analysis, we show that this hypothesis accounts for visual sensitivity to a large set of grayscale image statistics at a striking level of detail, and also identify the limits of the prediction. We define a 66-dimensional space of local grayscale light-intensity correlations, and measure the relevance of each direction to natural scenes. The 'variance is salience' hypothesis predicts that two-point correlations are most salient, and predicts their relative salience. We tested these predictions in a texture-segregation task using un-natural, synthetic textures. As predicted, correlations beyond second order are not salient, and predicted thresholds for over 300 second-order correlations match psychophysical thresholds closely (median fractional error <0.13).

The features that control discrimination of an isodipole texture pair.

Visual features such as edges and corners are carried by high-order statistics. Previous analysis of discrimination of isodipole textures, which isolate specific high-order statistics, demonstrates visual sensitivity to these statistics but stops short of analyzing the underlying computations. Here we use a new texture centroid paradigm to probe these computations. We focus on two canonical isodipole textures, the even and odd textures: any 2 × 2 block of even (odd) texture contains an even (odd) number of black (and white) checks. Each stimulus comprised a spatially random array of black-and-white texture-disks (background = mean gray) that varied in their fourth-order statistics. In the Even (Odd) condition, disks varied along the continuum between random coinflip texture and pure (highly structured) even (odd) target texture. The task was to mouse-click the centroid of the disk array, weighting each disk location by the target structure level of the disk-texture (ranging from 0 for coinflip to 1 for even or odd). For each of block-sizes S=2×2, 2 × 3, 2 × 4 and 3 × 3, a linear model was used to estimate the weight exerted on the subject's responses by the differently patterned blocks of size S. Only the results with 2 × 4 and 3 × 3 blocks were consistent with the data. In the Even condition, homogeneous blocks exerted the most weight; in the odd condition, block-pattern symmetry was important. These findings show that visual mechanisms sensitive to four-point correlations do not compute evenness or oddness per se, but rather are activated selectively by features whose frequency varies across isodipole textures.

Image segmentation driven by elements of form.

While luminance, contrast, orientation, and terminators are well-established features that are extracted in early visual processing and support the parsing of an image into its component regions, the role of more complex features, such as closure and convexity, is less clear. A main barrier in understanding the roles of such features is that manipulating their occurrence typically entails changes in the occurrence of more elementary features as well. To address this problem, we developed a set of synthetic visual textures, constructed by replacing the binary coloring of standard maximum-entropy textures with tokens (tiles) containing curved or angled elements. The tokens were designed so that there were no discontinuities at their edges, and so that changing the correlation structure of the underlying binary texture changed the shapes that were produced. The resulting textures were then used in psychophysical studies, demonstrating that the resulting feature differences sufficed to drive segmentation. However, in contrast to previous findings for lower-level features, sensitivities to increases and decreases of feature occurrence were unequal. Moreover, the texture-segregation response depended on the kind of token (curved vs. angular, filled-in vs. outlined), and not just on the correlation structure. Analysis of this dependence indicated that simple closed contours and convex elements suffice to drive image segmentation, in the absence of changes in lower-level cues.

Cortical Response to the Natural Speech Envelope Correlates with Neuroimaging Evidence of Cognition in Severe Brain Injury.

Recent studies identify severely brain-injured patients with limited or no behavioral responses who successfully perform functional magnetic resonance imaging (fMRI) or electroencephalogram (EEG) mental imagery tasks [1-5]. Such tasks are cognitively demanding [1]; accordingly, recent studies support that fMRI command following in brain-injured patients associates with preserved cerebral metabolism and preserved sleep-wake EEG [5, 6]. We investigated the use of an EEG response that tracks the natural speech envelope (NSE) of spoken language [7-22] in healthy controls and brain-injured patients (vegetative state to emergence from minimally conscious state). As audition is typically preserved after brain injury, auditory paradigms may be preferred in searching for covert cognitive function [23-25]. NSE measures are obtained by cross-correlating EEG with the NSE. We compared NSE latencies and amplitudes with and without consideration of fMRI assessments. NSE latencies showed significant and progressive delay across diagnostic categories. Patients who could carry out fMRI-based mental imagery tasks showed no statistically significant difference in NSE latencies relative to healthy controls; this subgroup included patients without behavioral command following. The NSE may stratify patients with severe brain injuries and identify those patients demonstrating "cognitive motor dissociation" (CMD) [26] who show only covert evidence of command following utilizing neuroimaging or electrophysiological methods that demand high levels of cognitive function. Thus, the NSE is a passive measure that may provide a useful screening tool to improve detection of covert cognition with fMRI or other methods and improve stratification of patients with disorders of consciousness in research studies.

Characterization of EEG signals revealing covert cognition in the injured brain.

See Boly and Laureys (doi:10.1093/brain/awy080) for a scientific commentary on this article.Patients with severe brain injury are difficult to assess and frequently subject to misdiagnosis. 'Cognitive motor dissociation' is a term used to describe a subset of such patients with preserved cognition as detected with neuroimaging methods but not evident in behavioural assessments. Unlike the locked-in state, cognitive motor dissociation after severe brain injury is prominently marked by concomitant injuries across the cerebrum in addition to limited or no motoric function. In the present study, we sought to characterize the EEG signals used as indicators of cognition in patients with disorders of consciousness and examine their reliability for potential future use to re-establish communication. We compared EEG-based assessments to the results of using similar methods with functional MRI. Using power spectral density analysis to detect EEG evidence of task performance (Two Group Test, P ≤ 0.05, with false discovery rate correction), we found evidence of the capacity to follow commands in 21 of 28 patients with severe brain injury and all 15 healthy individuals studied. We found substantial variability in the temporal and spatial characteristics of significant EEG signals among the patients in contrast to only modest variation in these domains across healthy controls; the majority of healthy controls showed suppression of either 8-12 Hz 'alpha' or 13-40 Hz 'beta' power during task performance, or both. Nine of the 21 patients with EEG evidence of command-following also demonstrated functional MRI evidence of command-following. Nine of the patients with command-following capacity demonstrated by EEG showed no behavioural evidence of a communication channel as detected by a standardized behavioural assessment, the Coma Recovery Scale - Revised. We further examined the potential contributions of fluctuations in arousal that appeared to co-vary with some patients' ability to reliably generate EEG signals in response to command. Five of nine patients with statistically indeterminate responses to one task tested showed a positive response after accounting for variations in overall background state (as visualized in the qualitative shape of the power spectrum) and grouping of trial runs with similar background state characteristics. Our findings reveal signal variations of EEG responses in patients with severe brain injuries and provide insight into the underlying physiology of cognitive motor dissociation. These results can help guide future efforts aimed at re-establishment of communication in such patients who will need customization for brain-computer interfaces.

Divergent neural responses to narrative speech in disorders of consciousness.

Objective: Clinical assessment of auditory attention in patients with disorders of consciousness is often limited by motor impairment. Here, we employ intersubject correlations among electroencephalography responses to naturalistic speech in order to assay auditory attention among patients and healthy controls. Methods: Electroencephalographic data were recorded from 20 subjects with disorders of consciousness and 14 healthy controls during of two narrative audio stimuli, presented both forwards and time-reversed. Intersubject correlation of evoked electroencephalography signals were calculated, comparing responses of both groups to those of the healthy control subjects. This analysis was performed blinded and subsequently compared to the diagnostic status of each patient based on the Coma Recovery Scale-Revised. Results: Subjects with disorders of consciousness exhibit significantly lower intersubject correlation than healthy controls during narrative speech. Additionally, while healthy subjects had higher intersubject correlation values in forwards versus backwards presentation, neural responses did not vary significantly with the direction of playback in subjects with disorders of consciousness. Increased intersubject correlation values in the backward speech condition were noted with improving disorder of consciousness diagnosis, both in cross-sectional analysis and in a subset of patients with longitudinal data. Interpretation: Intersubject correlation of neural responses to narrative speech audition differentiates healthy controls from patients and appears to index clinical diagnoses in disorders of consciousness.

Textures as Probes of Visual Processing.

Visual textures are a class of stimuli with properties that make them well suited for addressing general questions about visual function at the levels of behavior and neural mechanism. They have structure across multiple spatial scales, they put the focus on the inferential nature of visual processing, and they help bridge the gap between stimuli that are analytically convenient and the complex, naturalistic stimuli that have the greatest biological relevance. Key questions that are well suited for analysis via visual textures include the nature and structure of perceptual spaces, modulation of early visual processing by task, and the transformation of sensory stimuli into patterns of population activity that are relevant to perception.

Two representations of a high-dimensional perceptual space.

A perceptual space is a mental workspace of points in a sensory domain that supports similarity and difference judgments and enables further processing such as classification and naming. Perceptual spaces are present across sensory modalities; examples include colors, faces, auditory textures, and odors. Color is perhaps the best-studied perceptual space, but it is atypical in two respects. First, the dimensions of color space are directly linked to the three cone absorption spectra, but the dimensions of generic perceptual spaces are not as readily traceable to single-neuron properties. Second, generic perceptual spaces have more than three dimensions. This is important because representing each distinguishable point in a high-dimensional space by a separate neuron or population is unwieldy; combinatorial strategies may be needed to overcome this hurdle. To study the representation of a complex perceptual space, we focused on a well-characterized 10-dimensional domain of visual textures. Within this domain, we determine perceptual distances in a threshold task (segmentation) and a suprathreshold task (border salience comparison). In N=4 human observers, we find both quantitative and qualitative differences between these sets of measurements. Quantitatively, observers' segmentation thresholds were inconsistent with their uncertainty determined from border salience comparisons. Qualitatively, segmentation thresholds suggested that distances are determined by a coordinate representation with Euclidean geometry. Border salience comparisons, in contrast, indicated a global curvature of the space, and that distances are determined by activity patterns across broadly tuned elements. Thus, our results indicate two representations of this perceptual space, and suggest that they use differing combinatorial strategies. SIGNIFICANCE STATEMENT: To move from sensory signals to decisions and actions, the brain carries out a sequence of transformations. An important stage in this process is the construction of a "perceptual space" - an internal workspace of sensory information that captures similarities and differences, and enables further processing, such as classification and naming. Perceptual spaces for color, faces, visual and haptic textures and shapes, sounds, and odors (among others) are known to exist. How such spaces are represented is at present unknown. Here, using visual textures as a model, we investigate this. Psychophysical measurements suggest roles for two combinatorial strategies: one based on projections onto coordinate-like axes, and one based on patterns of activity across broadly tuned elements scattered throughout the space.

Executive attention deficits after traumatic brain injury reflect impaired recruitment of resources.

Deficits in attention are a common and devastating consequence of traumatic brain injury (TBI), leading to functional impairments, rehabilitation barriers, and long-term disability. While such deficits are well documented, little is known about their underlying pathophysiology hindering development of effective and targeted interventions. Here we evaluate the integrity of brain systems specific to attentional functions using quantitative assessments of electroencephalography recorded during performance of the Attention Network Test (ANT), a behavioral paradigm that separates alerting, orienting, and executive components of attention. We studied 13 patients, at least 6 months post-TBI with cognitive impairments, and 24 control subjects. Based on performance on the ANT, TBI subjects showed selective impairment in executive attention. In TBI subjects, principal component analysis combined with spectral analysis of the EEG after target appearance extracted a pattern of increased frontal midline theta power (2.5-7.5 Hz) and suppression of frontal beta power (12.5-22.5 Hz). Individual expression of this pattern correlated (r = - 0.67, p < 0.001) with executive attention impairment. The grading of this pattern of spatiotemporal dynamics with executive attention deficits reflects impaired recruitment of anterior forebrain resources following TBI; specifically, deafferentation and variable disfacilitation of medial frontal neuronal populations is proposed as the basis of our findings.

A perceptual space of local image statistics.

Local image statistics are important for visual analysis of textures, surfaces, and form. There are many kinds of local statistics, including those that capture luminance distributions, spatial contrast, oriented segments, and corners. While sensitivity to each of these kinds of statistics have been well-studied, much less is known about visual processing when multiple kinds of statistics are relevant, in large part because the dimensionality of the problem is high and different kinds of statistics interact. To approach this problem, we focused on binary images on a square lattice - a reduced set of stimuli which nevertheless taps many kinds of local statistics. In this 10-parameter space, we determined psychophysical thresholds to each kind of statistic (16 observers) and all of their pairwise combinations (4 observers). Sensitivities and isodiscrimination contours were consistent across observers. Isodiscrimination contours were elliptical, implying a quadratic interaction rule, which in turn determined ellipsoidal isodiscrimination surfaces in the full 10-dimensional space, and made predictions for sensitivities to complex combinations of statistics. These predictions, including the prediction of a combination of statistics that was metameric to random, were verified experimentally. Finally, check size had only a mild effect on sensitivities over the range from 2.8 to 14min, but sensitivities to second- and higher-order statistics was substantially lower at 1.4min. In sum, local image statistics form a perceptual space that is highly stereotyped across observers, in which different kinds of statistics interact according to simple rules.

Variance predicts salience in central sensory processing.

Information processing in the sensory periphery is shaped by natural stimulus statistics. In the periphery, a transmission bottleneck constrains performance; thus efficient coding implies that natural signal components with a predictably wider range should be compressed. In a different regime--when sampling limitations constrain performance--efficient coding implies that more resources should be allocated to informative features that are more variable. We propose that this regime is relevant for sensory cortex when it extracts complex features from limited numbers of sensory samples. To test this prediction, we use central visual processing as a model: we show that visual sensitivity for local multi-point spatial correlations, described by dozens of independently-measured parameters, can be quantitatively predicted from the structure of natural images. This suggests that efficient coding applies centrally, where it extends to higher-order sensory features and operates in a regime in which sensitivity increases with feature variability.

Preservation of electroencephalographic organization in patients with impaired consciousness and imaging-based evidence of command-following.

OBJECTIVE: Standard clinical characterization of patients with disorders of consciousness (DOC) relies on observation of motor output and may therefore lead to the misdiagnosis of vegetative state or minimally conscious state in patients with preserved cognition. We used conventional electroencephalographic (EEG) measures to assess a cohort of DOC patients with and without functional magnetic resonance imaging (fMRI)-based evidence of command-following, and correlated the findings with standard clinical behavioral evaluation and brain metabolic activity. METHODS: We enrolled 44 patients with severe brain injury. Behavioral diagnosis was established using standardized clinical assessments. Long-term EEG recordings were analyzed to determine wakeful background organization and presence of elements of sleep architecture. A subset of patients had fMRI testing of command-following using motor imagery paradigms (26 patients) and resting brain metabolism measurement using (18) fluorodeoxyglucose positron emission tomography (31 patients). RESULTS: All 4 patients with fMRI evidence of covert command-following consistently demonstrated well-organized EEG background during wakefulness, spindling activity during sleep, and relative preservation of cortical metabolic activity. In the entire cohort, EEG organization and overall brain metabolism showed no significant association with bedside behavioral testing, except in a few cases when EEG was severely abnormal. INTERPRETATION: These findings suggest that conventional EEG is a simple strategy that complements behavioral and imaging characterization of DOC patients. Preservation of specific EEG features may be used to assess the likelihood of unrecognized cognitive abilities in severely brain-injured patients with very limited or no motor responses.

Common resting brain dynamics indicate a possible mechanism underlying zolpidem response in severe brain injury.

Zolpidem produces paradoxical recovery of speech, cognitive and motor functions in select subjects with severe brain injury but underlying mechanisms remain unknown. In three diverse patients with known zolpidem responses we identify a distinctive pattern of EEG dynamics that suggests a mechanistic model. In the absence of zolpidem, all subjects show a strong low frequency oscillatory peak ∼6-10 Hz in the EEG power spectrum most prominent over frontocentral regions and with high coherence (∼0.7-0.8) within and between hemispheres. Zolpidem administration sharply reduces EEG power and coherence at these low frequencies. The ∼6-10 Hz activity is proposed to arise from intrinsic membrane properties of pyramidal neurons that are passively entrained across the cortex by locally-generated spontaneous activity. Activation by zolpidem is proposed to arise from a combination of initial direct drug effects on cortical, striatal, and thalamic populations and further activation of underactive brain regions induced by restoration of cognitively-mediated behaviors. DOI: http://dx.doi.org/10.7554/eLife.01157.001.