The lexical categorization model: A computational model of left ventral occipito-temporal cortex activation in visual word recognition.

To characterize the functional role of the left-ventral occipito-temporal cortex (lvOT) during reading in a quantitatively explicit and testable manner, we propose the lexical categorization model (LCM). The LCM assumes that lvOT optimizes linguistic processing by allowing fast meaning access when words are familiar and filtering out orthographic strings without meaning. The LCM successfully simulates benchmark results from functional brain imaging described in the literature. In a second evaluation, we empirically demonstrate that quantitative LCM simulations predict lvOT activation better than alternative models across three functional magnetic resonance imaging studies. We found that word-likeness, assumed as input into a lexical categorization process, is represented posteriorly to lvOT, whereas a dichotomous word/non-word output of the LCM could be localized to the downstream frontal brain regions. Finally, training the process of lexical categorization resulted in more efficient reading. In sum, we propose that word recognition in the ventral visual stream involves word-likeness extraction followed by lexical categorization before one can access word meaning.

Eye movements during text reading align with the rate of speech production.

Across languages, the speech signal is characterized by a predominant modulation of the amplitude spectrum between about 4.3 and 5.5 Hz, reflecting the production and processing of linguistic information chunks (syllables and words) every ~200 ms. Interestingly, ~200 ms is also the typical duration of eye fixations during reading. Prompted by this observation, we demonstrate that German readers sample written text at ~5 Hz. A subsequent meta-analysis of 142 studies from 14 languages replicates this result and shows that sampling frequencies vary across languages between 3.9 Hz and 5.2 Hz. This variation systematically depends on the complexity of the writing systems (character-based versus alphabetic systems and orthographic transparency). Finally, we empirically demonstrate a positive correlation between speech spectrum and eye movement sampling in low-skilled non-native readers, with tentative evidence from post hoc analysis suggesting the same relationship in low-skilled native readers. On the basis of this convergent evidence, we propose that during reading, our brain's linguistic processing systems imprint a preferred processing rate-that is, the rate of spoken language production and perception-onto the oculomotor system.

Predicting intelligence from brain gray matter volume.

A positive association between brain size and intelligence is firmly established, but whether region-specific anatomical differences contribute to general intelligence remains an open question. Results from voxel-based morphometry (VBM) - one of the most widely used morphometric methods - have remained inconclusive so far. Here, we applied cross-validated machine learning-based predictive modeling to test whether out-of-sample prediction of individual intelligence scores is possible on the basis of voxel-wise gray matter volume. Features were derived from structural magnetic resonance imaging data (N = 308) using (a) a purely data-driven method (principal component analysis) and (b) a domain knowledge-based approach (atlas parcellation). When using relative gray matter (corrected for total brain size), only the atlas-based approach provided significant prediction, while absolute gray matter (uncorrected) allowed for above-chance prediction with both approaches. Importantly, in all significant predictions, the absolute error was relatively high, i.e., greater than ten IQ points, and in the atlas-based models, the predicted IQ scores varied closely around the sample mean. This renders the practical value even of statistically significant prediction results questionable. Analyses based on the gray matter of functional brain networks yielded significant predictions for the fronto-parietal network and the cerebellum. However, the mean absolute errors were not reduced in contrast to the global models, suggesting that general intelligence may be related more to global than region-specific differences in gray matter volume. More generally, our study highlights the importance of predictive statistical analysis approaches for clarifying the neurobiological bases of intelligence and provides important suggestions for future research using predictive modeling.

An orthographic prediction error as the basis for efficient visual word recognition.

Most current models assume that the perceptual and cognitive processes of visual word recognition and reading operate upon neuronally coded domain-general low-level visual representations - typically oriented line representations. We here demonstrate, consistent with neurophysiological theories of Bayesian-like predictive neural computations, that prior visual knowledge of words may be utilized to 'explain away' redundant and highly expected parts of the visual percept. Subsequent processing stages, accordingly, operate upon an optimized representation of the visual input, the orthographic prediction error, highlighting only the visual information relevant for word identification. We show that this optimized representation is related to orthographic word characteristics, accounts for word recognition behavior, and is processed early in the visual processing stream, i.e., in V4 and before 200 â€‹ms after word-onset. Based on these findings, we propose that prior visual-orthographic knowledge is used to optimize the representation of visually presented words, which in turn allows for highly efficient reading processes.

Traces of Meaning Itself: Encoding Distributional Word Vectors in Brain Activity.

How is semantic information stored in the human mind and brain? Some philosophers and cognitive scientists argue for vectorial representations of concepts, where the meaning of a word is represented as its position in a high-dimensional neural state space. At the intersection of natural language processing and artificial intelligence, a class of very successful distributional word vector models has developed that can account for classic EEG findings of language, that is, the ease versus difficulty of integrating a word with its sentence context. However, models of semantics have to account not only for context-based word processing, but should also describe how word meaning is represented. Here, we investigate whether distributional vector representations of word meaning can model brain activity induced by words presented without context. Using EEG activity (event-related brain potentials) collected while participants in two experiments (English and German) read isolated words, we encoded and decoded word vectors taken from the family of prediction-based Word2vec algorithms. We found that, first, the position of a word in vector space allows the prediction of the pattern of corresponding neural activity over time, in particular during a time window of 300 to 500 ms after word onset. Second, distributional models perform better than a human-created taxonomic baseline model (WordNet), and this holds for several distinct vector-based models. Third, multiple latent semantic dimensions of word meaning can be decoded from brain activity. Combined, these results suggest that empiricist, prediction-based vectorial representations of meaning are a viable candidate for the representational architecture of human semantic knowledge.

Neurophysiological markers of ADHD symptoms in typically-developing children.

Children with attention-deficit/hyperactivity disorder (ADHD) are characterized by symptoms of inattention, impulsivity, and hyperactivity. Neurophysiological correlates of ADHD include changes in the P3 component of event-related brain potentials (ERPs). Motivated by recent advances towards a more dimensional understanding of ADHD, we investigate whether ADHD-related ERP markers relate to continuous variations in attention and executive functioning also in typically-developing children. ERPs were measured while 31 school children (9-11 years) completed an adapted version of the Continuous Performance Task that additionally to inhibitory processes also isolates effects of physical stimulus salience. Children with higher levels of parent-reported ADHD symptoms did not differ in task performance, but exhibited smaller P3 amplitudes related to stimulus salience. Furthermore, ADHD symptoms were associated with the variability of neural responses over time: Children with higher levels of ADHD symptoms demonstrated lower variability in inhibition- and salience-related P3 amplitudes. No effects were observed for ERP latencies and the salience-related N2. By demonstrating that ADHD-associated neurophysiological mechanisms of inhibition and salience processing covary with attention and executive functioning in a children community sample, our study provides neurophysiological support for dimensional models of ADHD. Also, temporal variability in event-related potentials is highlighted as additional indicator of ADHD requiring further investigation.

Finding the P3 in the P600: Decoding shared neural mechanisms of responses to syntactic violations and oddball targets.

The P600 Event-Related Brain Potential, elicited by syntactic violations in sentences, is generally interpreted as indicating language-specific structural/combinatorial processing, with far-reaching implications for models of language. P600 effects are also often taken as evidence for language-like grammars in non-linguistic domains like music or arithmetic. An alternative account, however, interprets the P600 as a P3, a domain-general brain response to salience. Using time-generalized multivariate pattern analysis, we demonstrate that P3 EEG patterns, elicited in a visual Oddball experiment, account for the P600 effect elicited in a syntactic violation experiment: P3 pattern-trained MVPA can classify P600 trials just as well as P600-trained ones. A second study replicates and generalizes this finding, and demonstrates its specificity by comparing it to face- and semantic mismatch-associated EEG responses. These results indicate that P3 and P600 share neural patterns to a substantial degree, calling into question the interpretation of P600 as a language-specific brain response and instead strengthening its association with the P3. More generally, our data indicate that observing P600-like brain responses provides no direct evidence for the presence of language-like grammars, in language or elsewhere.

Cluster-based permutation tests of MEG/EEG data do not establish significance of effect latency or location.

Cluster-based permutation tests are gaining an almost universal acceptance as inferential procedures in cognitive neuroscience. They elegantly handle the multiple comparisons problem in high-dimensional magnetoencephalographic and EEG data. Unfortunately, the power of this procedure comes hand in hand with the allure for unwarranted interpretations of the inferential output, the most prominent of which is the overestimation of the temporal, spatial, and frequency precision of statistical claims. This leads researchers to statements about the onset or offset of a certain effect that is not supported by the permutation test. In this article, we outline problems and common pitfalls of using and interpreting cluster-based permutation tests. We illustrate these with simulated data in order to promote a more intuitive understanding of the method. We hope that raising awareness about these issues will be beneficial to common scientific practices, while at the same time increasing the popularity of cluster-based permutation procedures.

Improving free-viewing fixation-related EEG potentials with continuous-time regression.

BACKGROUND: In the analysis of combined ET-EEG data, there are several issues with estimating FRPs by averaging. Neural responses associated with fixations will likely overlap with one another in the EEG recording and neural responses change as a function of eye movement characteristics. Especially in tasks that do not constrain eye movements in any way, these issues can become confounds. NEW METHOD: Here, we propose the use of regression based estimates as an alternative to averaging. Multiple regression can disentangle different influences on the EEG and correct for overlap. It thereby accounts for potential confounds in a way that averaging cannot. Specifically, we test the applicability of the rERP framework, as proposed by Smith and Kutas (2015b), (2017), or Sassenhagen (2018) to combined eye tracking and EEG data from a visual search and a scene memorization task. RESULTS: Results show that the method successfully estimates eye movement related confounds in real experimental data, so that these potential confounds can be accounted for when estimating experimental effects. COMPARISON WITH EXISTING METHODS: The rERP method successfully corrects for overlapping neural responses in instances where averaging does not. As a consequence, baselining can be applied without risking distortions. By estimating a known experimental effect, we show that rERPs provide an estimate with less variance and more accuracy than averaged FRPs. The method therefore provides a practically feasible and favorable alternative to averaging. CONCLUSIONS: We conclude that regression based ERPs provide novel opportunities for estimating fixation related EEG in free-viewing experiments.

No evidence from MVPA for different processes underlying the N300 and N400 incongruity effects in object-scene processing.

Attributing meaning to diverse visual input is a core feature of human cognition. Violating environmental expectations (e.g., a toothbrush in the fridge) induces a late event-related negativity of the event-related potential/ERP. This N400 ERP has not only been linked to the semantic processing of language, but also to objects and scenes. Inconsistent object-scene relationships are additionally associated with an earlier negative deflection of the EEG signal between 250 and 350 ms. This N300 is hypothesized to reflect pre-semantic perceptual processes. To investigate whether these two components are truly separable or if the early object-scene integration activity (250-350 ms) shares certain levels of processing with the late neural correlates of meaning processing (350-500 ms), we used time-resolved multivariate pattern analysis (MVPA) where a classifier trained at one time point in a trial (e.g., during the N300 time window) is tested at every other time point (i.e., including the N400 time window). Forty participants were presented with semantic inconsistencies, in which an object was inconsistent with a scene's meaning. Replicating previous findings, our manipulation produced significant N300 and N400 deflections. MVPA revealed above chance decoding performance for classifiers trained during time points of the N300 component and tested during later time points of the N400, and vice versa. This provides no evidence for the activation of two separable neurocognitive processes following the violation of context-dependent predictions in visual scene perception. Our data supports the early appearance of high-level, context-sensitive processes in visual cognition.

Time-generalized multivariate analysis of EEG responses reveals a cascading architecture of semantic mismatch processing.

Event-related brain potentials have a strong impact on neurocognitive models, as they inform about the temporal sequence of cognitive processes. Nevertheless, their value for deciding among alternative cognitive architectures is partly limited by component overlap and the possibility of ambiguity regarding component identity. Here, we apply temporally-generalized multivariate pattern analysis - a recently-proposed machine learning method capable of tracking the evolution of neurocognitive processes over time - to constrain possible alternative architectures underlying the processing of semantic incongruency in sentences. In a spoken sentence paradigm, we replicate established N400/P600 correlates of semantic mismatch. Time-generalized decoding indicates that early vs. late mismatch-sensitive processes are (i) distinct in their neural substrate, arguing against recurrent or latency-shifted single process architectures, and (ii) partially overlapping in time, inconsistent with predictions of strictly serial models. These results are in accordance with an incremental-cascading neurocognitive organization of semantic mismatch processing. We propose time-generalized multivariate decoding as a valuable tool for neurocognitive language studies.

The oddball effect on P3 disappears when feature relevance or feature-response mappings are unknown.

The P3b component of human event-related EEG potentials is larger with rare than frequent task-relevant stimuli. In a previous study, this oddball effect was much reduced when stimulus-response (S-R) mappings were still undefined at stimulus presentation (being later provided by response prompts). This reduction may reflect P3b's dependence on transmitted information which might be any relevant information (informational value hypothesis) or, more specifically, information about how to respond (S-R link hypothesis). To distinguish between these two hypotheses and clarify their differences from classical stimulus evaluation hypothesis, we added a second dimension by presenting colored letters, with both colors and letters varying between a rare and a frequent alternative. Response prompts, presented half a second later, were, in different blocks, constant or variable across trials with respect to S-R mapping and with respect to the relevant dimension (color or letter). With partial information, when only one of these two factors is known at stimulus presentation (by being constant across trials), the hypotheses differ in their predictions. The oddball effect will be abolished according to S-R link hypothesis because knowledge of both factors is needed to determine the response, but will only be reduced according to informational value hypothesis and be fully maintained according to stimulus evaluation hypothesis. In fact, oddball effects only occurred with knowledge of both factors, i.e., if both the relevant dimension and its mapping to responses were constant across trials. These results confirm the preeminent role of knowledge about responses for eliciting P3.

Domain-general neural correlates of dependency formation: Using complex tones to simulate language.

There is an ongoing debate whether the P600 event-related potential component following syntactic anomalies reflects syntactic processes per se, or if it is an instance of the P300, a domain-general ERP component associated with attention and cognitive reorientation. A direct comparison of both components is challenging because of the huge discrepancy in experimental designs and stimulus choice between language and 'classic' P300 experiments. In the present study, we develop a new approach to mimic the interplay of sequential position as well as categorical and relational information in natural language syntax (word category and agreement) in a non-linguistic target detection paradigm using musical instruments. Participants were instructed to (covertly) detect target tones which were defined by instrument change and pitch rise between subsequent tones at the last two positions of four-tone sequences. We analysed the EEG using event-related averaging and time-frequency decomposition. Our results show striking similarities to results obtained from linguistic experiments. We found a P300 that showed sensitivity to sequential position and a late positivity sensitive to stimulus type and position. A time-frequency decomposition revealed significant effects of sequential position on the theta band and a significant influence of stimulus type on the delta band. Our results suggest that the detection of non-linguistic targets defined via complex feature conjunctions in the present study and the detection of syntactic anomalies share the same underlying processes: attentional shift and memory based matching processes that act upon multi-feature conjunctions. We discuss the results as supporting domain-general accounts of the P600 during natural language comprehension.

A common misapplication of statistical inference: Nuisance control with null-hypothesis significance tests.

Experimental research on behavior and cognition frequently rests on stimulus or subject selection where not all characteristics can be fully controlled, even when attempting strict matching. For example, when contrasting patients to controls, variables such as intelligence or socioeconomic status are often correlated with patient status. Similarly, when presenting word stimuli, variables such as word frequency are often correlated with primary variables of interest. One procedure very commonly employed to control for such nuisance effects is conducting inferential tests on confounding stimulus or subject characteristics. For example, if word length is not significantly different for two stimulus sets, they are considered as matched for word length. Such a test has high error rates and is conceptually misguided. It reflects a common misunderstanding of statistical tests: interpreting significance not to refer to inference about a particular population parameter, but about 1. the sample in question, 2. the practical relevance of a sample difference (so that a nonsignificant test is taken to indicate evidence for the absence of relevant differences). We show inferential testing for assessing nuisance effects to be inappropriate both pragmatically and philosophically, present a survey showing its high prevalence, and briefly discuss an alternative in the form of regression including nuisance variables.

The P600 as a correlate of ventral attention network reorientation.

When, during language processing, a reader or listener is confronted with a structurally deviant phrase, this typically elicits a late positive ERP deflection (P600). The P600 is often understood as a correlate of structural analysis. This assumption has informed a number of neurocognitive models of language. However, the P600 strongly resembles the P3, likely a more general electrophysiological correlate of reorientation behaviour supported by noradrenergic input to the ventral attention network/VAN. Some researchers have proposed that the P600 is an instance of the P3, not a distinct component reflecting the analysis of structured inputs. Here, we tested the P600-as-P3 hypothesis by estimating the alignment of the P600 elicited in a visual sentence processing task to simultaneously collected behavioural measures. A similar analysis was undertaken for a P3 elicited in a separate non-linguistic (face detection) task. Since the P3 is usually aligned to reaction time/RT, the same should hold for the P600; a failure to find RT alignment of the P600 would pose a problem for the P600-as-P3 hypothesis. In contrast, RT alignment of the P600 would associate it with the well-established VAN/Locus Coeruleus - Noradrenaline - Network subserving cortical reorientation. We failed to falsify the hypothesis of RT alignment. Secondary measures, while less unambiguous, were more in agreement with the P600-as-P3 hypothesis. We interpret our results as corroborating the hypothesis that the P600 is a P3, in that it shows that the P600 is RT-aligned. This perspective is unpredicted by an account of the P600 as indexing high-level processing.

The P600-as-P3 hypothesis revisited: single-trial analyses reveal that the late EEG positivity following linguistically deviant material is reaction time aligned.

The P600, a late positive ERP component following linguistically deviant stimuli, is commonly seen as indexing structural, high-level processes, e.g. of linguistic (re)analysis. It has also been identified with the P3 (P600-as-P3 hypothesis), which is thought to reflect a systemic neuromodulator release facilitating behavioural shifts and is usually response time aligned. We investigated single-trial alignment of the P600 to response, a critical prediction of the P600-as-P3 hypothesis. Participants heard sentences containing morphosyntactic and semantic violations and responded via a button press. The elicited P600 was perfectly response aligned, while an N400 following semantic deviations was stimulus aligned. This is, to our knowledge, the first single-trial analysis of language processing data using within-sentence behavioural responses as temporal covariates. Results support the P600-as-P3 perspective and thus constitute a step towards a neurophysiological grounding of language-related ERPs.

Cross-linguistic variation in the neurophysiological response to semantic processing: evidence from anomalies at the borderline of awareness.

The N400 event-related brain potential (ERP) has played a major role in the examination of how the human brain processes meaning. For current theories of the N400, classes of semantic inconsistencies which do not elicit N400 effects have proven particularly influential. Semantic anomalies that are difficult to detect are a case in point ("borderline anomalies", e.g. "After an air crash, where should the survivors be buried?"), engendering a late positive ERP response but no N400 effect in English (Sanford, Leuthold, Bohan, & Sanford, 2011). In three auditory ERP experiments, we demonstrate that this result is subject to cross-linguistic variation. In a German version of Sanford and colleagues' experiment (Experiment 1), detected borderline anomalies elicited both N400 and late positivity effects compared to control stimuli or to missed borderline anomalies. Classic easy-to-detect semantic (non-borderline) anomalies showed the same pattern as in English (N400 plus late positivity). The cross-linguistic difference in the response to borderline anomalies was replicated in two additional studies with a slightly modified task (Experiment 2a: German; Experiment 2b: English), with a reliable LANGUAGE×ANOMALY interaction for the borderline anomalies confirming that the N400 effect is subject to systematic cross-linguistic variation. We argue that this variation results from differences in the language-specific default weighting of top-down and bottom-up information, concluding that N400 amplitude reflects the interaction between the two information sources in the form-to-meaning mapping.