Multisyllabic decoding achievement and relation to vocabulary at the end of elementary school.

Multisyllabic decoding poses a significant challenge to upper elementary grade readers. The purposes of this study were to (a) examine the reliability and validity of a classroom assessment, the Multisyllabic Decoding Inventory (MDI); (b) describe fourth- and fifth-grade students' decoding of multisyllabic words in relation to their semantic difficulty (age of acquisition ratings); (c) investigate which aspects of word knowledge (word recognition and decoding skill, vocabulary knowledge, and morphological knowledge) predict real word and nonword reading for multisyllabic words; and (d) determine how student word knowledge and semantic difficulty of words jointly affect the odds of accurately recognizing a multisyllabic word. We found that (a) the MDI demonstrated acceptable internal consistency reliability and concurrent validity with standardized measures of word recognition and oral reading fluency; (b) students demonstrated strong performance in reading multisyllabic words and nonwords, but words with higher age of acquisition were less frequently recognized; (c) multisyllabic word reading was predicted by word recognition and decoding skill, vocabulary knowledge, and morphological knowledge, whereas multisyllabic nonword reading was predicted by decoding skills and morphological knowledge only; and (d) grade level, word recognition and decoding skill, and vocabulary at the student level increased the odds of recognizing a multisyllabic word correctly, whereas a word's age of acquisition rating decreased the odds of recognizing a multisyllabic word correctly. The results suggest that students in the upper elementary grades may benefit from multisyllabic decoding instruction that integrates decoding and vocabulary strategies.

Symbolic and non-symbolic representations of numerical zero in the human brain.

Representing the quantity zero as a symbolic concept is considered a unique achievement of abstract human thought.1,2 To conceptualize zero, one must abstract away from the (absence of) sensory evidence to construct a representation of numerical absence: creating "something" out of "nothing."2,3,4 Previous investigations of the neural representation of natural numbers reveal distinct numerosity-selective neural populations that overlap in their tuning curves with adjacent numerosities.5,6 Importantly, a component of this neural code is thought to be invariant across non-symbolic and symbolic numerical formats.7,8,9,10,11 Although behavioral evidence indicates that zero occupies a place at the beginning of this mental number line,12,13,14 in humans zero is also associated with unique behavioral and developmental profiles compared to natural numbers,4,15,16,17 suggestive of a distinct neural basis for zero. We characterized the neural representation of zero in the human brain by employing two qualitatively different numerical tasks18,19 in concert with magnetoencephalography (MEG) recordings. We assay both neural representations of non-symbolic numerosities (dot patterns), including zero (empty sets), and symbolic numerals, including symbolic zero. Our results reveal that neural representations of zero are situated along a graded neural number line shared with other natural numbers. Notably, symbolic representations of zero generalized to predict non-symbolic empty sets. We go on to localize abstract representations of numerical zero to posterior association cortex, extending the purview of parietal cortex in human numerical cognition to encompass representations of zero.10,20.

Similar neural states, but dissimilar decoding patterns for motor control in parietal cortex.

Discrete neural states are associated with reaching movements across the fronto-parietal network. Here, the Hidden Markov Model (HMM) applied to spiking activity of the somato-motor parietal area PE revealed a sequence of states similar to those of the contiguous visuomotor areas PEc and V6A. Using a coupled clustering and decoding approach, we proved that these neural states carried spatiotemporal information regarding behaviour in all three posterior parietal areas. However, comparing decoding accuracy, PE was less informative than V6A and PEc. In addition, V6A outperformed PEc in target inference, indicating functional differences among the parietal areas. To check the consistency of these differences, we used both a supervised and an unsupervised variant of the HMM, and compared its performance with two more common classifiers, Support Vector Machine and Long-Short Term Memory. The differences in decoding between areas were invariant to the algorithm used, still showing the dissimilarities found with HMM, thus indicating that these dissimilarities are intrinsic in the information encoded by parietal neurons. These results highlight that, when decoding from the parietal cortex, for example, in brain machine interface implementations, attention should be paid in selecting the most suitable source of neural signals, given the great heterogeneity of this cortical sector.

Applying HMMs to spiking activity recorded from the somato-motor parietal area PE revealed discrete neural states related to reaching movements. These states were extremely similar to those present in the neighbouring visuomotor areas PEc and V6A. Our decoding approach showed that these states conveyed spatiotemporal behaviour information across all three posterior parietal areas. However, decoding accuracy was lower in PE compared to V6A and PEc, with V6A excelling in target inference. These differences held true even when changing the decoding algorithm, indicating intrinsic dissimilarities in information encoding by parietal different areas. These findings highlight the importance of selecting the appropriate neural signal sources in applications such as brain machine interfaces and pave the way for further investigation of the nontrivial diversity within the parietal cortex.

Reading acquisition among students in Grades 1-3 with intellectual disabilities in Sweden.

This study investigates the reading performance of younger students with intellectual disabilities to gain insight into their needs in reading education. Participants were 428 students in Grades 1 to 3 in Sweden. They performed LegiLexi tests measuring pre-reading skills, decoding and reading comprehension based on the model of Simple View of Reading. Results demonstrate a great variation in reading acquisition among students. Some students are able to decode single words and read shorter texts with comprehension already in Grade 1. Other students still struggle with learning letters and developing phonological awareness in Grade 3. According to their longitudinal data over grades, results show that most students progress in pre-reading skills, decoding, and reading comprehension. Hence, assessing reading skills among students with intellectual disabilities in Grades 1-3 using tools aligned with the Simple View of Reading seems applicable and informative for teachers. This study underscores the significance of informed instructional practices for empowering these students in reading education.

The State-Dependent Channel with a Rate-Limited Cribbing Helper.

The capacity of a memoryless state-dependent channel is derived for a setting in which the encoder is provided with rate-limited assistance from a cribbing helper that observes the state sequence causally and the past channel inputs strictly causally. Said cribbing may increase capacity but not to the level achievable by a message-cognizant helper.

Research on a soft saturation nonlinear SSVEP signal feature extraction algorithm.

Brain-computer interfaces (BCIs) based on steady-state visual evoked potentials (SSVEP) have received widespread attention due to their high information transmission rate, high accuracy, and rich instruction set. However, the performance of its identification methods strongly depends on the amount of calibration data for within-subject classification. Some studies use deep learning (DL) algorithms for inter-subject classification, which can reduce the calculation process, but there is still much room for improvement in performance compared with intra-subject classification. To solve these problems, an efficient SSVEP signal recognition deep learning network model e-SSVEPNet based on the soft saturation nonlinear module is proposed in this paper. The soft saturation nonlinear module uses a similar exponential calculation method for output when it is less than zero, improving robustness to noise. Under the conditions of the SSVEP data set, two sliding time window lengths (1 s and 0.5 s), and three training data sizes, this paper evaluates the proposed network model and compares it with other traditional and deep learning model baseline methods. The experimental results of the nonlinear module were classified and compared. A large number of experimental results show that the proposed network has the highest average accuracy of intra-subject classification on the SSVEP data set, improves the performance of SSVEP signal classification and recognition, and has higher decoding accuracy under short signals, so it has huge potential ability to realize high-speed SSVEP-based for BCI.

Decoding EEG for optimizing naturalistic memory.

BACKGROUND: Spectral features of human electroencephalographic (EEG) recordings during learning predict subsequent recall variability. METHODS: Capitalizing on these fluctuating neural features, we develop a non-invasive closed-loop (NICL) system for real-time optimization of human learning. Participants play a virtual navigation and memory game; recording multi-session data across days allowed us to build participant-specific classification models of recall success. In subsequent closed-loop sessions, our platform manipulated the timing of memory encoding, selectively presenting items during periods of predicted good or poor memory function based on EEG features decoded in real time. RESULTS: We observed greater memory modulation (difference between recall rates when presenting items during predicted good vs. poor learning periods) for participants with higher out-of-sample classification accuracy. COMPARISON WITH EXISTING METHODS: This study demonstrates greater-than-chance memory decoding from EEG recordings in a naturalistic virtual navigation task with greater real-world validity than basic word-list recall paradigms. Here we modulate memory by timing stimulus presentation based on noninvasive scalp EEG recordings, whereas prior closed-loop studies for memory improvement involved intracranial recordings and direct electrical stimulation. Other noninvasive studies have investigated the use of neurofeedback or remedial study for memory improvement. CONCLUSION: These findings present a proof-of-concept for using non-invasive closed-loop technology to optimize human learning and memory through principled stimulus timing, but only in those participants for whom classifiers reliably predict out-of-sample memory function.

Dynamic decoding and dual synthetic data for automatic correction of grammar in low-resource scenario.

Grammar error correction systems are pivotal in the field of natural language processing (NLP), with a primary focus on identifying and correcting the grammatical integrity of written text. This is crucial for both language learning and formal communication. Recently, neural machine translation (NMT) has emerged as a promising approach in high demand. However, this approach faces significant challenges, particularly the scarcity of training data and the complexity of grammar error correction (GEC), especially for low-resource languages such as Indonesian. To address these challenges, we propose InSpelPoS, a confusion method that combines two synthetic data generation methods: the Inverted Spellchecker and Patterns+POS. Furthermore, we introduce an adapted seq2seq framework equipped with a dynamic decoding method and state-of-the-art Transformer-based neural language models to enhance the accuracy and efficiency of GEC. The dynamic decoding method is capable of navigating the complexities of GEC and correcting a wide range of errors, including contextual and grammatical errors. The proposed model leverages the contextual information of words and sentences to generate a corrected output. To assess the effectiveness of our proposed framework, we conducted experiments using synthetic data and compared its performance with existing GEC systems. The results demonstrate a significant improvement in the accuracy of Indonesian GEC compared to existing methods.

Decoding cognition in neurodevelopmental, psychiatric and neurological conditions with multivariate pattern analysis of EEG data.

Multivariate pattern analysis (MVPA) of electroencephalographic (EEG) data represents a revolutionary approach to investigate how the brain encodes information. By considering complex interactions among spatio-temporal features at the individual level, MVPA overcomes the limitations of univariate techniques, which often fail to account for the significant inter- and intra-individual neural variability. This is particularly relevant when studying clinical populations, and therefore MVPA of EEG data has recently started to be employed as a tool to study cognition in brain disorders. Here, we review the insights offered by this methodology in the study of anomalous patterns of neural activity in conditions such as autism, ADHD, schizophrenia, dyslexia, neurological and neurodegenerative disorders, within different cognitive domains (perception, attention, memory, consciousness). Despite potential drawbacks that should be attentively addressed, these studies reveal a peculiar sensitivity of MVPA in unveiling dysfunctional and compensatory neurocognitive dynamics of information processing, which often remain blind to traditional univariate approaches. Such higher sensitivity in characterizing individual neurocognitive profiles can provide unique opportunities to optimise assessment and promote personalised interventions.

Mental state decoders: game-changers or wishful thinking?

Decoding mental and perceptual states using fMRI has become increasingly popular over the past two decades, with numerous highly-cited studies published in high-profile journals. Nevertheless, what have we learned from these decoders? In this opinion, we argue that fMRI-based decoders are not neurophysiologically informative and are not, and likely cannot be, applicable to real-world decision-making. The former point stems from the fact that decoding models cannot disentangle neural mechanisms from their epiphenomena. The latter point stems from both logical and ethical constraints. Constructing decoders requires precious time and resources that should instead be directed toward scientific endeavors more likely to yield meaningful scientific progress.

Explainable deep-learning framework: decoding brain states and prediction of individual performance in false-belief task at early childhood stage.

Decoding of cognitive states aims to identify individuals' brain states and brain fingerprints to predict behavior. Deep learning provides an important platform for analyzing brain signals at different developmental stages to understand brain dynamics. Due to their internal architecture and feature extraction techniques, existing machine-learning and deep-learning approaches are suffering from low classification performance and explainability issues that must be improved. In the current study, we hypothesized that even at the early childhood stage (as early as 3-years), connectivity between brain regions could decode brain states and predict behavioral performance in false-belief tasks. To this end, we proposed an explainable deep learning framework to decode brain states (Theory of Mind and Pain states) and predict individual performance on ToM-related false-belief tasks in a developmental dataset. We proposed an explainable spatiotemporal connectivity-based Graph Convolutional Neural Network (Ex-stGCNN) model for decoding brain states. Here, we consider a developmental dataset, N = 155 (122 children; 3-12 yrs and 33 adults; 18-39 yrs), in which participants watched a short, soundless animated movie, shown to activate Theory-of-Mind (ToM) and pain networs. After scanning, the participants underwent a ToM-related false-belief task, leading to categorization into the pass, fail, and inconsistent groups based on performance. We trained our proposed model using Functional Connectivity (FC) and Inter-Subject Functional Correlations (ISFC) matrices separately. We observed that the stimulus-driven feature set (ISFC) could capture ToM and Pain brain states more accurately with an average accuracy of 94%, whereas it achieved 85% accuracy using FC matrices. We also validated our results using five-fold cross-validation and achieved an average accuracy of 92%. Besides this study, we applied the SHapley Additive exPlanations (SHAP) approach to identify brain fingerprints that contributed the most to predictions. We hypothesized that ToM network brain connectivity could predict individual performance on false-belief tasks. We proposed an Explainable Convolutional Variational Auto-Encoder (Ex-Convolutional VAE) model to predict individual performance on false-belief tasks and trained the model using FC and ISFC matrices separately. ISFC matrices again outperformed the FC matrices in prediction of individual performance. We achieved 93.5% accuracy with an F1-score of 0.94 using ISFC matrices and achieved 90% accuracy with an F1-score of 0.91 using FC matrices.

Duplicated Heschl's gyrus associations with phonological decoding.

The reason(s) for why a complete duplication of the left hemisphere Heschl's gyrus (HG) has been observed in people with reading disability are unclear. This study was designed to replicate and advance understanding of the HG and phonological decoding association, as well as test competing hypotheses that this HG duplication association is specifically localized to the HG or could be due to co-occurring atypical development of other brain regions that support reading and language development. Participants were selected on the basis of having a duplicated left hemisphere HG (N = 96) or a single HG (N = 96) and matched according to age, sex, and research site in this multi-site study. Duplicated and single HG morphology specific templates were created to determine the extent to which HG sizes were related to phonological decoding within each HG morphology group. The duplicated HG group had significantly lower phonological decoding (F = 4.48, p = 0.04) but not verbal IQ (F = 1.39, p = 0.41) compared to the single HG group. In addition, larger HG were significantly associated with lower phonological decoding in the duplicated HG group, with effects driven by the size of the lateral HG after controlling for age, sex, research site, and handedness (ps < 0.05). Brain regions that exhibited structural covariance with HG did not clearly explain the HG and phonological decoding associations. Together, the results suggest that presence of a duplicated HG indicates some risk for lower phonological decoding ability compared to verbal IQ, but the reason(s) for this association remain unclear.

Confirmation Bias through Selective Use of Evidence in Human Cortex.

Decision-makers often process new evidence selectively, depending on their current beliefs about the world. We asked whether such confirmation biases result from biases in the encoding of sensory evidence in the brain, or alternatively in the utilization of encoded evidence for behavior. Human participants estimated the source of a sequence of visual-spatial evidence samples while we measured cortical population activity with magnetoencephalography (MEG). Halfway through the sequence, participants were prompted to judge the more likely source category. Their processing of subsequent evidence depended on its consistency with the previously chosen category, but the encoding of evidence in cortical activity did not. Instead, the encoded evidence in parietal and primary visual cortex contributed less to the estimation report when that evidence was inconsistent with the previous choice. We conclude that confirmation bias originates from the way in which decision-makers utilize information encoded in the brain. This provides room for deliberative control.

What does decoding from the PFC reveal about consciousness?

Disputes between rival theories of consciousness have often centered on whether perceptual contents can be decoded from the prefrontal cortex (PFC). Failures to decode from the PFC are taken to challenge 'cognitive' theories of consciousness such as the global workspace theory and higher-order monitoring theories, and decoding successes have been taken to confirm these theories. However, PFC decoding shows both too much and too little. Too much because cognitive theories of consciousness do not need PFC rerepresentation of perceptual contents since pointers to perceptual representations suffice. Too little because there is evidence that PFC decoding of perceptual content reflects postperceptual cognitive representation, such as thoughts that have those perceptual contents rather than conscious percepts.

Retrieving and reconstructing conceptually similar images from fMRI with latent diffusion models and a neuro-inspired brain decoding model.

Objective.Brain decoding is a field of computational neuroscience that aims to infer mental states or internal representations of perceptual inputs from measurable brain activity. This study proposes a novel approach to brain decoding that relies on semantic and contextual similarity.Approach.We use several functional magnetic resonance imaging (fMRI) datasets of natural images as stimuli and create a deep learning decoding pipeline inspired by the bottom-up and top-down processes in human vision. Our pipeline includes a linear brain-to-feature model that maps fMRI activity to semantic visual stimuli features. We assume that the brain projects visual information onto a space that is homeomorphic to the latent space of last layer of a pretrained neural network, which summarizes and highlights similarities and differences between concepts. These features are categorized in the latent space using a nearest-neighbor strategy, and the results are used to retrieve images or condition a generative latent diffusion model to create novel images.Main results.We demonstrate semantic classification and image retrieval on three different fMRI datasets: Generic Object Decoding (vision perception and imagination), BOLD5000, and NSD. In all cases, a simple mapping between fMRI and a deep semantic representation of the visual stimulus resulted in meaningful classification and retrieved or generated images. We assessed quality using quantitative metrics and a human evaluation experiment that reproduces the multiplicity of conscious and unconscious criteria that humans use to evaluate image similarity. Our method achieved correct evaluation in over 80% of the test set.Significance.Our study proposes a novel approach to brain decoding that relies on semantic and contextual similarity. The results demonstrate that measurable neural correlates can be linearly mapped onto the latent space of a neural network to synthesize images that match the original content. These findings have implications for both cognitive neuroscience and artificial intelligence.

Overcoming the High Error Rate of Composite DNA Letters-Based Digital Storage through Soft-Decision Decoding.

Composite DNA letters, by merging all four DNA nucleotides in specified ratios, offer a pathway to substantially increase the logical density of DNA digital storage (DDS) systems. However, these letters are susceptible to nucleotide errors and sampling bias, leading to a high letter error rate, which complicates precise data retrieval and augments reading expenses. To address this, Derrick-cp is introduced as an innovative soft-decision decoding algorithm tailored for DDS utilizing composite letters. Derrick-cp capitalizes on the distinctive error sensitivities among letters to accurately predict and rectify letter errors, thus enhancing the error-correcting performance of Reed-Solomon codes beyond traditional hard-decision decoding limits. Through comparative analyses in the existing dataset and simulated experiments, Derrick-cp's superiority is validated, notably halving the sequencing depth requirement and slashing costs by up to 22% against conventional hard-decision strategies. This advancement signals Derrick-cp's significant role in elevating both the precision and cost-efficiency of composite letter-based DDS.

Spiking generative adversarial network with attention scoring decoding.

Generative models based on neural networks present a substantial challenge within deep learning. As it stands, such models are primarily limited to the domain of artificial neural networks. Spiking neural networks, as the third generation of neural networks, offer a closer approximation to brain-like processing due to their rich spatiotemporal dynamics. However, generative models based on spiking neural networks are not well studied. Particularly, previous works on generative adversarial networks based on spiking neural networks are conducted on simple datasets and do not perform well. In this work, we pioneer constructing a spiking generative adversarial network capable of handling complex images and having higher performance. Our first task is to identify the problems of out-of-domain inconsistency and temporal inconsistency inherent in spiking generative adversarial networks. We address these issues by incorporating the Earth-Mover distance and an attention-based weighted decoding method, significantly enhancing the performance of our algorithm across several datasets. Experimental results reveal that our approach outperforms existing methods on the MNIST, FashionMNIST, CIFAR10, and CelebA. In addition to our examination of static datasets, this study marks our inaugural investigation into event-based data, through which we achieved noteworthy results. Moreover, compared with hybrid spiking generative adversarial networks, where the discriminator is an artificial analog neural network, our methodology demonstrates closer alignment with the information processing patterns observed in the mouse. Our code can be found at https://github.com/Brain-Cog-Lab/sgad.

KEMoS: A knowledge-enhanced multi-modal summarizing framework for Chinese online meetings.

The demand for "online meetings" and "collaborative office work" keeps surging recently, producing an abundant amount of relevant data. How to provide participants with accurate and fast summarizing service has attracted extensive attention. Existing meeting summarizing models overlook the utilization of multi-modal information and the information offsetting during summarizing. In this paper, we develop a knowledge-enhanced multi-modal summarizing framework. Firstly, we construct a three-layer multi-modal meeting knowledge graph, including basic, knowledge, and multi-modal layer, to integrate meeting information thoroughly. Then, we raise a topic-based hierarchical clustering approach, which considers information entropy and difference simultaneously, to capture the semantic evolution of meetings. Next, we devise a multi-modal enhanced encoding strategy, including a sentence-level cross-modal encoder, a joint loss function, and a knowledge graph embedding module, to learn the meeting and topic-level presentations. Finally, when generating summaries, we design a topic-enhanced decoding strategy for the Transformer decoder which mitigates semantic offsetting with the aid of topic information. Extensive experiments show that our proposed work consistently outperforms state-of-the-art solutions on the Chinese meeting dataset, where the ROUGE-1, ROUGE-2, and ROUGE-L are 49.98%, 21.03%, and 32.03% respectively.

'Are you even listening?' - EEG-based decoding of absolute auditory attention to natural speech.

Objective.In this study, we use electroencephalography (EEG) recordings to determine whether a subject is actively listening to a presented speech stimulus. More precisely, we aim to discriminate between an active listening condition, and a distractor condition where subjects focus on an unrelated distractor task while being exposed to a speech stimulus. We refer to this task as absolute auditory attention decoding.Approach.We re-use an existing EEG dataset where the subjects watch a silent movie as a distractor condition, and introduce a new dataset with two distractor conditions (silently reading a text and performing arithmetic exercises). We focus on two EEG features, namely neural envelope tracking (NET) and spectral entropy (SE). Additionally, we investigate whether the detection of such an active listening condition can be combined with a selective auditory attention decoding (sAAD) task, where the goal is to decide to which of multiple competing speakers the subject is attending. The latter is a key task in so-called neuro-steered hearing devices that aim to suppress unattended audio, while preserving the attended speaker.Main results.Contrary to a previous hypothesis of higher SE being related with actively listening rather than passively listening (without any distractors), we find significantly lower SE in the active listening condition compared to the distractor conditions. Nevertheless, the NET is consistently significantly higher when actively listening. Similarly, we show that the accuracy of a sAAD task improves when evaluating the accuracy only on the highest NET segments. However, the reverse is observed when evaluating the accuracy only on the lowest SE segments.Significance.We conclude that the NET is more reliable for decoding absolute auditory attention as it is consistently higher when actively listening, whereas the relation of the SE between actively and passively listening seems to depend on the nature of the distractor.

Decoding visual brain representations from electroencephalography through knowledge distillation and latent diffusion models.

Decoding visual representations from human brain activity has emerged as a thriving research domain, particularly in the context of brain-computer interfaces. Our study presents an innovative method that employs knowledge distillation to train an EEG classifier and reconstruct images from the ImageNet and THINGS-EEG 2 datasets using only electroencephalography (EEG) data from participants who have viewed the images themselves (i.e. "brain decoding"). We analyzed EEG recordings from 6 participants for the ImageNet dataset and 10 for the THINGS-EEG 2 dataset, exposed to images spanning unique semantic categories. These EEG readings were converted into spectrograms, which were then used to train a convolutional neural network (CNN), integrated with a knowledge distillation procedure based on a pre-trained Contrastive Language-Image Pre-Training (CLIP)-based image classification teacher network. This strategy allowed our model to attain a top-5 accuracy of 87%, significantly outperforming a standard CNN and various RNN-based benchmarks. Additionally, we incorporated an image reconstruction mechanism based on pre-trained latent diffusion models, which allowed us to generate an estimate of the images that had elicited EEG activity. Therefore, our architecture not only decodes images from neural activity but also offers a credible image reconstruction from EEG only, paving the way for, e.g., swift, individualized feedback experiments.