Discrimination of textures with spatial correlations and multiple gray levels.

Analysis of visual texture is important for many key steps in early vision. We study visual sensitivity to image statistics in three families of textures that include multiple gray levels and correlations in two spatial dimensions. Sensitivities to positive and negative correlations are approximately independent of correlation sign, and signals from different kinds of correlations combine quadratically. We build a computational model, fully constrained by prior studies of sensitivity to uncorrelated textures and black-and-white textures with spatial correlations. The model accounts for many features of the new data, including sign-independence, quadratic combination, and the dependence on gray-level distribution.

Characterization of EEG signals revealing covert cognition in the injured brain.

See Boly and Laureys (doi:10.1093/brain/awy080) for a scientific commentary on this article.Patients with severe brain injury are difficult to assess and frequently subject to misdiagnosis. 'Cognitive motor dissociation' is a term used to describe a subset of such patients with preserved cognition as detected with neuroimaging methods but not evident in behavioural assessments. Unlike the locked-in state, cognitive motor dissociation after severe brain injury is prominently marked by concomitant injuries across the cerebrum in addition to limited or no motoric function. In the present study, we sought to characterize the EEG signals used as indicators of cognition in patients with disorders of consciousness and examine their reliability for potential future use to re-establish communication. We compared EEG-based assessments to the results of using similar methods with functional MRI. Using power spectral density analysis to detect EEG evidence of task performance (Two Group Test, P ≤ 0.05, with false discovery rate correction), we found evidence of the capacity to follow commands in 21 of 28 patients with severe brain injury and all 15 healthy individuals studied. We found substantial variability in the temporal and spatial characteristics of significant EEG signals among the patients in contrast to only modest variation in these domains across healthy controls; the majority of healthy controls showed suppression of either 8-12 Hz 'alpha' or 13-40 Hz 'beta' power during task performance, or both. Nine of the 21 patients with EEG evidence of command-following also demonstrated functional MRI evidence of command-following. Nine of the patients with command-following capacity demonstrated by EEG showed no behavioural evidence of a communication channel as detected by a standardized behavioural assessment, the Coma Recovery Scale - Revised. We further examined the potential contributions of fluctuations in arousal that appeared to co-vary with some patients' ability to reliably generate EEG signals in response to command. Five of nine patients with statistically indeterminate responses to one task tested showed a positive response after accounting for variations in overall background state (as visualized in the qualitative shape of the power spectrum) and grouping of trial runs with similar background state characteristics. Our findings reveal signal variations of EEG responses in patients with severe brain injuries and provide insight into the underlying physiology of cognitive motor dissociation. These results can help guide future efforts aimed at re-establishment of communication in such patients who will need customization for brain-computer interfaces.

Preservation of electroencephalographic organization in patients with impaired consciousness and imaging-based evidence of command-following.

OBJECTIVE: Standard clinical characterization of patients with disorders of consciousness (DOC) relies on observation of motor output and may therefore lead to the misdiagnosis of vegetative state or minimally conscious state in patients with preserved cognition. We used conventional electroencephalographic (EEG) measures to assess a cohort of DOC patients with and without functional magnetic resonance imaging (fMRI)-based evidence of command-following, and correlated the findings with standard clinical behavioral evaluation and brain metabolic activity. METHODS: We enrolled 44 patients with severe brain injury. Behavioral diagnosis was established using standardized clinical assessments. Long-term EEG recordings were analyzed to determine wakeful background organization and presence of elements of sleep architecture. A subset of patients had fMRI testing of command-following using motor imagery paradigms (26 patients) and resting brain metabolism measurement using (18) fluorodeoxyglucose positron emission tomography (31 patients). RESULTS: All 4 patients with fMRI evidence of covert command-following consistently demonstrated well-organized EEG background during wakefulness, spindling activity during sleep, and relative preservation of cortical metabolic activity. In the entire cohort, EEG organization and overall brain metabolism showed no significant association with bedside behavioral testing, except in a few cases when EEG was severely abnormal. INTERPRETATION: These findings suggest that conventional EEG is a simple strategy that complements behavioral and imaging characterization of DOC patients. Preservation of specific EEG features may be used to assess the likelihood of unrecognized cognitive abilities in severely brain-injured patients with very limited or no motor responses.

Mean-field modeling of thalamocortical dynamics and a model-driven approach to EEG analysis.

Higher brain function depends on task-dependent information flow between cortical regions. Converging lines of evidence suggest that interactions between cortical regions and the central thalamus play a key role in establishing the dynamic patterns of functional connectivity that normally support these processes. In patients with chronic disturbances of cognitive function due to severe brain injury, dysfunction of this circuitry likely plays a crucial role in pathogenesis. However, assaying thalamocortical interactions is challenging even in healthy subjects and more so in severely impaired patients. To approach this problem, we apply a dynamical-systems approach to motivate an analysis of the electroencephalogram (EEG). We begin with a model for a single thalamocortical module [Robinson PA, Rennie CJ, Rowe DL (2002) Phys Rev E Stat Nonlin Soft Matter Phys 65:041924; Robinson PA, Rennie CJ, Wright JJ, Bourke PD (1998) Phys Rev E Stat Nonlin Soft Matter Phys 58:3557-3571]. When two such modules interact via shared thalamic inhibition, multistable behavior emerges; each mode is characterized by a different pattern of coherence between cortical regions. This observation suggests that changing patterns of cortical coherence are a hallmark of normal thalamocortical dynamics. In a preliminary study, we test this idea by analyzing the EEG of a patient with chronic brain injury, who has a marked improvement in behavior and frontal brain metabolism in response to zolpidem. The analysis shows that following zolpidem administration, changing patterns of coherence are identified between the frontal lobes and between frontal and distant brain regions. These observations support the role of the central thalamus in the organization of patterns of cortical interactions and suggest how indexes of thalamocortical dynamics can be extracted from the EEG.

Perception of second- and third-order orientation signals and their interactions.

Orientation signals, which are crucial to many aspects of visual function, are more complex and varied in the natural world than in the stimuli typically used for laboratory investigation. Gratings and lines have a single orientation, but in natural stimuli, local features have multiple orientations, and multiple orientations can occur even at the same location. Moreover, orientation cues can arise not only from pairwise spatial correlations, but from higher-order ones as well. To investigate these orientation cues and how they interact, we examined segmentation performance for visual textures in which the strengths of different kinds of orientation cues were varied independently, while controlling potential confounds such as differences in luminance statistics. Second-order cues (the kind present in gratings) at different orientations are largely processed independently: There is no cancellation of positive and negative signals at orientations that differ by 45°. Third-order orientation cues are readily detected and interact only minimally with second-order cues. However, they combine across orientations in a different way: Positive and negative signals largely cancel if the orientations differ by 90°. Two additional elements are superimposed on this picture. First, corners play a special role. When second-order orientation cues combine to produce corners, they provide a stronger signal for texture segregation than can be accounted for by their individual effects. Second, while the object versus background distinction does not influence processing of second-order orientation cues, this distinction influences the processing of third-order orientation cues.

Low-level language processing in brain-injured patients.

Assessing cognitive function-especially language processing-in severely brain-injured patients is critical for prognostication, care, and development of communication devices (e.g. brain-computer interfaces). In patients with diminished motor function, language processing has been probed using EEG measures of command-following in motor imagery tasks. While such tests eliminate the need for motor response, they require sustained attention. However, passive listening tasks, with an EEG response measure can reduce both motor and attentional demands. These considerations motivated the development of two assays of low-level language processing-identification of differential phoneme-class responses and tracking of the natural speech envelope. This cross-sectional study looks at a cohort of 26 severely brain-injured patient subjects and 10 healthy controls. Patients' level of function was assessed via the coma recovery scale-revised at the bedside. Patients were also tested for command-following via EEG and/or MRI assays of motor imagery. For the present investigation, EEG was recorded while presenting a 148 s audio clip of Alice in Wonderland. Time-locked EEG responses to phoneme classes were extracted and compared to determine a differential phoneme-class response. Tracking of the natural speech envelope was assessed from the same recordings by cross-correlating the EEG response with the speech envelope. In healthy controls, the dynamics of the two measures were temporally similar but spatially different: a central parieto-occipital component of differential phoneme-class response was absent in the natural speech envelope response. The differential phoneme-class response was present in all patient subjects, including the six classified as vegetative state/unresponsive wakefulness syndrome by behavioural assessment. However, patient subjects with evidence of language processing either by behavioural assessment or motor imagery tests had an early bilateral response in the first 50 ms that was lacking in patient subjects without any evidence of language processing. The natural speech envelope tracking response was also present in all patient subjects and responses in the first 100 ms distinguished patient subjects with evidence of language processing. Specifically, patient subjects with evidence of language processing had a more global response in the first 100 ms whereas those without evidence of language processing had a frontopolar response in that period. In summary, we developed two passive EEG-based methods to probe low-level language processing in severely brain-injured patients. In our cohort, both assays showed a difference between patient subjects with evidence of command-following and those with no evidence of command-following: a more prominent early bilateral response component.

Local image statistics: maximum-entropy constructions and perceptual salience.

The space of visual signals is high-dimensional and natural visual images have a highly complex statistical structure. While many studies suggest that only a limited number of image statistics are used for perceptual judgments, a full understanding of visual function requires analysis not only of the impact of individual image statistics, but also, how they interact. In natural images, these statistical elements (luminance distributions, correlations of low and high order, edges, occlusions, etc.) are intermixed, and their effects are difficult to disentangle. Thus, there is a need for construction of stimuli in which one or more statistical elements are introduced in a controlled fashion, so that their individual and joint contributions can be analyzed. With this as motivation, we present algorithms to construct synthetic images in which local image statistics--including luminance distributions, pair-wise correlations, and higher-order correlations--are explicitly specified and all other statistics are determined implicitly by maximum-entropy. We then apply this approach to measure the sensitivity of the human visual system to local image statistics and to sample their interactions.

Functional recursion of orientation cues in figure-ground separation.

Visual texture is an important cue to figure-ground organization. While processing of texture differences is a prerequisite for the use of this cue to extract figure-ground organization, these stages are distinct processes. One potential indicator of this distinction is the possibility that texture statistics play a different role in the figure vs. in the ground. To determine whether this is the case, we probed figure-ground processing with a family of local image statistics that specified textures that varied in the strength and spatial scale of structure, and the extent to which features are oriented. For image statistics that generated approximately isotropic textures, the threshold for identification of figure-ground structure was determined by the difference in correlation strength in figure vs. ground, independent of whether the correlations were present in figure, ground, or both. However, for image statistics with strong orientation content, thresholds were up to two times higher for correlations in the ground, vs. the figure. This held equally for texture-defined objects with convex or concave boundaries, indicating that these threshold differences are driven by border ownership, not boundary shape. Similar threshold differences were found for presentation times ranging from 125 to 500 ms. These findings identify a qualitative difference in how texture is used for figure-ground analysis, vs. texture discrimination. Additionally, it reveals a functional recursion: texture differences are needed to identify tentative boundaries and consequent scene organization into figure and ground, but then scene organization modifies sensitivity to texture differences according to the figure-ground assignment.

Electrophysiological correlates of thalamocortical function in acute severe traumatic brain injury.

Tools assaying the neural networks that modulate consciousness may facilitate tracking of recovery after acute severe brain injury. The ABCD framework classifies resting-state EEG into categories reflecting levels of thalamocortical network function that correlate with outcome in post-cardiac arrest coma. In this longitudinal cohort study, we applied the ABCD framework to 20 patients with acute severe traumatic brain injury requiring intensive care (12 of whom were also studied at ≥6-months post-injury) and 16 healthy controls. We tested four hypotheses: 1) EEG ABCD classifications are spatially heterogeneous and temporally variable; 2) ABCD classifications improve longitudinally, commensurate with the degree of behavioral recovery; 3) ABCD classifications correlate with behavioral level of consciousness; and 4) the Coma Recovery Scale-Revised arousal facilitation protocol yields improved ABCD classifications. Channel-level EEG power spectra were classified based on spectral peaks within pre-defined frequency bands: 'A' = no peaks above delta (<4 Hz) range (complete thalamocortical disruption); 'B' = theta (4-8 Hz) peak (severe thalamocortical disruption); 'C' = theta and beta (13-24 Hz) peaks (moderate thalamocortical disruption); or 'D' = alpha (8-13 Hz) and beta peaks (normal thalamocortical function). Acutely, 95% of patients demonstrated 'D' signals in at least one channel but exhibited within-session temporal variability and spatial heterogeneity in the proportion of different channel-level ABCD classifications. By contrast, healthy participants and patients at follow-up consistently demonstrated signals corresponding to intact thalamocortical network function. Patients demonstrated longitudinal improvement in ABCD classifications (p < .05) and ABCD classification distinguished patients with and without command-following in the subacute-to-chronic phase of recovery (p < .01). In patients studied acutely, ABCD classifications improved after the Coma Recovery Scale-Revised arousal facilitation protocol (p < .05) but did not correspond with behavioral level of consciousness. These findings support the use of the ABCD framework to characterize channel-level EEG dynamics and track fluctuations in functional thalamocortical network integrity in spatial detail.

Visual Search for Circumscribed Interests in Autism Is Similar to That of Neurotypical Individuals.

Intense interests are a core symptom of autism spectrum disorders (ASD) and can be all-encompassing for affected individuals. This observation raises the hypothesis that intense interests in ASD are related to pervasive changes in visual processing for objects within that category, including visual search. We assayed visual processing with two novel tasks, targeting category search and exemplar search. For each task, three kinds of stimuli were used: faces, houses, and images personalized to each participant's interest. 25 children and adults with ASD were compared to 25 neurotypical (NT) children and adults. We found no differences in either visual search task between ASD and NT controls for interests. Thus, pervasive alterations in perception are not likely to account for ASD behavioral symptoms.

Efficient coding of natural scene statistics predicts discrimination thresholds for grayscale textures.

Previously, in Hermundstad et al., 2014, we showed that when sampling is limiting, the efficient coding principle leads to a 'variance is salience' hypothesis, and that this hypothesis accounts for visual sensitivity to binary image statistics. Here, using extensive new psychophysical data and image analysis, we show that this hypothesis accounts for visual sensitivity to a large set of grayscale image statistics at a striking level of detail, and also identify the limits of the prediction. We define a 66-dimensional space of local grayscale light-intensity correlations, and measure the relevance of each direction to natural scenes. The 'variance is salience' hypothesis predicts that two-point correlations are most salient, and predicts their relative salience. We tested these predictions in a texture-segregation task using un-natural, synthetic textures. As predicted, correlations beyond second order are not salient, and predicted thresholds for over 300 second-order correlations match psychophysical thresholds closely (median fractional error <0.13).

The features that control discrimination of an isodipole texture pair.

Visual features such as edges and corners are carried by high-order statistics. Previous analysis of discrimination of isodipole textures, which isolate specific high-order statistics, demonstrates visual sensitivity to these statistics but stops short of analyzing the underlying computations. Here we use a new texture centroid paradigm to probe these computations. We focus on two canonical isodipole textures, the even and odd textures: any 2 × 2 block of even (odd) texture contains an even (odd) number of black (and white) checks. Each stimulus comprised a spatially random array of black-and-white texture-disks (background = mean gray) that varied in their fourth-order statistics. In the Even (Odd) condition, disks varied along the continuum between random coinflip texture and pure (highly structured) even (odd) target texture. The task was to mouse-click the centroid of the disk array, weighting each disk location by the target structure level of the disk-texture (ranging from 0 for coinflip to 1 for even or odd). For each of block-sizes S=2×2, 2 × 3, 2 × 4 and 3 × 3, a linear model was used to estimate the weight exerted on the subject's responses by the differently patterned blocks of size S. Only the results with 2 × 4 and 3 × 3 blocks were consistent with the data. In the Even condition, homogeneous blocks exerted the most weight; in the odd condition, block-pattern symmetry was important. These findings show that visual mechanisms sensitive to four-point correlations do not compute evenness or oddness per se, but rather are activated selectively by features whose frequency varies across isodipole textures.

Image segmentation driven by elements of form.

While luminance, contrast, orientation, and terminators are well-established features that are extracted in early visual processing and support the parsing of an image into its component regions, the role of more complex features, such as closure and convexity, is less clear. A main barrier in understanding the roles of such features is that manipulating their occurrence typically entails changes in the occurrence of more elementary features as well. To address this problem, we developed a set of synthetic visual textures, constructed by replacing the binary coloring of standard maximum-entropy textures with tokens (tiles) containing curved or angled elements. The tokens were designed so that there were no discontinuities at their edges, and so that changing the correlation structure of the underlying binary texture changed the shapes that were produced. The resulting textures were then used in psychophysical studies, demonstrating that the resulting feature differences sufficed to drive segmentation. However, in contrast to previous findings for lower-level features, sensitivities to increases and decreases of feature occurrence were unequal. Moreover, the texture-segregation response depended on the kind of token (curved vs. angular, filled-in vs. outlined), and not just on the correlation structure. Analysis of this dependence indicated that simple closed contours and convex elements suffice to drive image segmentation, in the absence of changes in lower-level cues.

Cortical Response to the Natural Speech Envelope Correlates with Neuroimaging Evidence of Cognition in Severe Brain Injury.

Recent studies identify severely brain-injured patients with limited or no behavioral responses who successfully perform functional magnetic resonance imaging (fMRI) or electroencephalogram (EEG) mental imagery tasks [1-5]. Such tasks are cognitively demanding [1]; accordingly, recent studies support that fMRI command following in brain-injured patients associates with preserved cerebral metabolism and preserved sleep-wake EEG [5, 6]. We investigated the use of an EEG response that tracks the natural speech envelope (NSE) of spoken language [7-22] in healthy controls and brain-injured patients (vegetative state to emergence from minimally conscious state). As audition is typically preserved after brain injury, auditory paradigms may be preferred in searching for covert cognitive function [23-25]. NSE measures are obtained by cross-correlating EEG with the NSE. We compared NSE latencies and amplitudes with and without consideration of fMRI assessments. NSE latencies showed significant and progressive delay across diagnostic categories. Patients who could carry out fMRI-based mental imagery tasks showed no statistically significant difference in NSE latencies relative to healthy controls; this subgroup included patients without behavioral command following. The NSE may stratify patients with severe brain injuries and identify those patients demonstrating "cognitive motor dissociation" (CMD) [26] who show only covert evidence of command following utilizing neuroimaging or electrophysiological methods that demand high levels of cognitive function. Thus, the NSE is a passive measure that may provide a useful screening tool to improve detection of covert cognition with fMRI or other methods and improve stratification of patients with disorders of consciousness in research studies.

VEPs elicited by local correlations and global symmetry: characteristics and interactions.

Psychophysical and fMRI studies have indicated that visual processing of global symmetry has distinctive scaling properties, and proceeds more slowly than analysis of contrast, spatial frequency, and texture. We therefore undertook a visual evoked potential (VEP) study to directly compare the dynamics of symmetry and texture processing, and to determine the extent to which they interact. Stimuli consisted of interchange between structured and random black-and-white checkerboard stimuli. For symmetry, structured stimuli were colored with 2-fold symmetry (horizontal or vertical mirror), 4-fold symmetry (both mirror axes), and 8-fold symmetry (oblique mirror axes added). For texture, structured stimuli were colored according to the "even" isodipole texture [Julesz, B., Gilbert, E. N., & Victor, J. D. (1978). Visual discrimination of textures with identical third-order statistics. Biological Cybernetics, 31, 137-140]. Thus, all stimuli had the same contrast, and check size, but differed substantially in correlation structure. To separate components of the VEP related to symmetry and texture from components that could be generated by local luminance and contrast changes, we extracted the odd-harmonic components of the VEP (recorded at C(z)-O(z), C(z)-O(1), C(z)-O(2), C(z)-P(z)) elicited by structured-random interchange. Responses to symmetry were largest for the 8-fold patterns, and progressively smaller for 4-fold, vertical, and horizontal symmetry patterns. Eightfold patterns were therefore used in the remainder of the study. The symmetry response is shifted to larger checks and lower temporal frequencies compared to the response to texture, and its temporal tuning is broader. Processing of symmetry makes use of neural mechanisms with larger receptive fields, and slower, more sustained temporal tuning characteristics than those involved in the analysis of texture. Sparse stimuli were used to dissociate check size and check density. VEP responses to sparse symmetry stimuli showed that there is no difference between first- and second-order symmetry for densities less than 12.5%. We discuss these findings in relation to local and global visual processes.

Textures as Probes of Visual Processing.

Visual textures are a class of stimuli with properties that make them well suited for addressing general questions about visual function at the levels of behavior and neural mechanism. They have structure across multiple spatial scales, they put the focus on the inferential nature of visual processing, and they help bridge the gap between stimuli that are analytically convenient and the complex, naturalistic stimuli that have the greatest biological relevance. Key questions that are well suited for analysis via visual textures include the nature and structure of perceptual spaces, modulation of early visual processing by task, and the transformation of sensory stimuli into patterns of population activity that are relevant to perception.

Two representations of a high-dimensional perceptual space.

A perceptual space is a mental workspace of points in a sensory domain that supports similarity and difference judgments and enables further processing such as classification and naming. Perceptual spaces are present across sensory modalities; examples include colors, faces, auditory textures, and odors. Color is perhaps the best-studied perceptual space, but it is atypical in two respects. First, the dimensions of color space are directly linked to the three cone absorption spectra, but the dimensions of generic perceptual spaces are not as readily traceable to single-neuron properties. Second, generic perceptual spaces have more than three dimensions. This is important because representing each distinguishable point in a high-dimensional space by a separate neuron or population is unwieldy; combinatorial strategies may be needed to overcome this hurdle. To study the representation of a complex perceptual space, we focused on a well-characterized 10-dimensional domain of visual textures. Within this domain, we determine perceptual distances in a threshold task (segmentation) and a suprathreshold task (border salience comparison). In N=4 human observers, we find both quantitative and qualitative differences between these sets of measurements. Quantitatively, observers' segmentation thresholds were inconsistent with their uncertainty determined from border salience comparisons. Qualitatively, segmentation thresholds suggested that distances are determined by a coordinate representation with Euclidean geometry. Border salience comparisons, in contrast, indicated a global curvature of the space, and that distances are determined by activity patterns across broadly tuned elements. Thus, our results indicate two representations of this perceptual space, and suggest that they use differing combinatorial strategies. SIGNIFICANCE STATEMENT: To move from sensory signals to decisions and actions, the brain carries out a sequence of transformations. An important stage in this process is the construction of a "perceptual space" - an internal workspace of sensory information that captures similarities and differences, and enables further processing, such as classification and naming. Perceptual spaces for color, faces, visual and haptic textures and shapes, sounds, and odors (among others) are known to exist. How such spaces are represented is at present unknown. Here, using visual textures as a model, we investigate this. Psychophysical measurements suggest roles for two combinatorial strategies: one based on projections onto coordinate-like axes, and one based on patterns of activity across broadly tuned elements scattered throughout the space.

Divergent neural responses to narrative speech in disorders of consciousness.

Objective: Clinical assessment of auditory attention in patients with disorders of consciousness is often limited by motor impairment. Here, we employ intersubject correlations among electroencephalography responses to naturalistic speech in order to assay auditory attention among patients and healthy controls. Methods: Electroencephalographic data were recorded from 20 subjects with disorders of consciousness and 14 healthy controls during of two narrative audio stimuli, presented both forwards and time-reversed. Intersubject correlation of evoked electroencephalography signals were calculated, comparing responses of both groups to those of the healthy control subjects. This analysis was performed blinded and subsequently compared to the diagnostic status of each patient based on the Coma Recovery Scale-Revised. Results: Subjects with disorders of consciousness exhibit significantly lower intersubject correlation than healthy controls during narrative speech. Additionally, while healthy subjects had higher intersubject correlation values in forwards versus backwards presentation, neural responses did not vary significantly with the direction of playback in subjects with disorders of consciousness. Increased intersubject correlation values in the backward speech condition were noted with improving disorder of consciousness diagnosis, both in cross-sectional analysis and in a subset of patients with longitudinal data. Interpretation: Intersubject correlation of neural responses to narrative speech audition differentiates healthy controls from patients and appears to index clinical diagnoses in disorders of consciousness.