Editorial Topical Collection: "Explainable and Augmented Machine Learning for Biosignals and Biomedical Images".

Machine learning (ML) is a well-known subfield of artificial intelligence (AI) that aims at developing algorithms and statistical models able to empower computer systems to automatically adapt to a specific task through experience or learning from data [...].

Brain-Inspired Spatio-Temporal Associative Memories for Neuroimaging Data Classification: EEG and fMRI.

Humans learn from a lot of information sources to make decisions. Once this information is learned in the brain, spatio-temporal associations are made, connecting all these sources (variables) in space and time represented as brain connectivity. In reality, to make a decision, we usually have only part of the information, either as a limited number of variables, limited time to make the decision, or both. The brain functions as a spatio-temporal associative memory. Inspired by the ability of the human brain, a brain-inspired spatio-temporal associative memory was proposed earlier that utilized the NeuCube brain-inspired spiking neural network framework. Here we applied the STAM framework to develop STAM for neuroimaging data, on the cases of EEG and fMRI, resulting in STAM-EEG and STAM-fMRI. This paper showed that once a NeuCube STAM classification model was trained on a complete spatio-temporal EEG or fMRI data, it could be recalled using only part of the time series, or/and only part of the used variables. We evaluated both temporal and spatial association and generalization accuracy accordingly. This was a pilot study that opens the field for the development of classification systems on other neuroimaging data, such as longitudinal MRI data, trained on complete data but recalled on partial data. Future research includes STAM that will work on data, collected across different settings, in different labs and clinics, that may vary in terms of the variables and time of data collection, along with other parameters. The proposed STAM will be further investigated for early diagnosis and prognosis of brain conditions and for diagnostic/prognostic marker discovery.

RNA-sequencing of peripheral whole blood of individuals at ultra-high-risk for psychosis - A longitudinal perspective.

BACKGROUND: The peripheral blood is an attractive source of prognostic biomarkers for psychosis conversion. There is limited research on the transcriptomic changes associated with psychosis conversion in the peripheral whole blood. STUDY DESIGN: We performed RNA-sequencing of peripheral whole blood from 65 ultra-high-risk (UHR) participants and 70 healthy control participants recruited in the Longitudinal Youth-at-Risk Study (LYRIKS) cohort. 13 UHR participants converted in the study duration. Samples were collected at 3 timepoints, at 12-months interval across a 2-year period. We examined whether the genes differential with psychosis conversion contain schizophrenia risk loci. We then examined the functional ontologies and GWAS associations of the differential genes. We also identified the overlap between differentially expressed genes across different comparisons. STUDY RESULTS: Genes containing schizophrenia risk loci were not differentially expressed in the peripheral whole blood in psychosis conversion. The differentially expressed genes in psychosis conversion are enriched for ontologies associated with cellular replication. The differentially expressed genes in psychosis conversion are associated with non-neurological GWAS phenotypes reported to be perturbed in schizophrenia and psychosis but not schizophrenia and psychosis phenotypes themselves. We found minimal overlap between the genes differential with psychosis conversion and the genes that are differential between pre-conversion and non-conversion samples. CONCLUSION: The associations between psychosis conversion and peripheral blood-based biomarkers are likely to be indirect. Further studies to elucidate the mechanism behind potential indirect associations are needed.

Filter and Wrapper Stacking Ensemble (FWSE): a robust approach for reliable biomarker discovery in high-dimensional omics data.

Selecting informative features, such as accurate biomarkers for disease diagnosis, prognosis and response to treatment, is an essential task in the field of bioinformatics. Medical data often contain thousands of features and identifying potential biomarkers is challenging due to small number of samples in the data, method dependence and non-reproducibility. This paper proposes a novel ensemble feature selection method, named Filter and Wrapper Stacking Ensemble (FWSE), to identify reproducible biomarkers from high-dimensional omics data. In FWSE, filter feature selection methods are run on numerous subsets of the data to eliminate irrelevant features, and then wrapper feature selection methods are applied to rank the top features. The method was validated on four high-dimensional medical datasets related to mental illnesses and cancer. The results indicate that the features selected by FWSE are stable and statistically more significant than the ones obtained by existing methods while also demonstrating biological relevance. Furthermore, FWSE is a generic method, applicable to various high-dimensional datasets in the fields of machine intelligence and bioinformatics.

Automated Road Defect and Anomaly Detection for Traffic Safety: A Systematic Review.

Recently, there has been a substantial increase in the development of sensor technology. As enabling factors, computer vision (CV) combined with sensor technology have made progress in applications intended to mitigate high rates of fatalities and the costs of traffic-related injuries. Although past surveys and applications of CV have focused on subareas of road hazards, there is yet to be one comprehensive and evidence-based systematic review that investigates CV applications for Automated Road Defect and Anomaly Detection (ARDAD). To present ARDAD's state-of-the-art, this systematic review is focused on determining the research gaps, challenges, and future implications from selected papers (N = 116) between 2000 and 2023, relying primarily on Scopus and Litmaps services. The survey presents a selection of artefacts, including the most popular open-access datasets (D = 18), research and technology trends that with reported performance can help accelerate the application of rapidly advancing sensor technology in ARDAD and CV. The produced survey artefacts can assist the scientific community in further improving traffic conditions and safety.

Investigation of social and cognitive predictors in non-transition ultra-high-risk' individuals for psychosis using spiking neural networks.

Finding predictors of social and cognitive impairment in non-transition Ultra-High-Risk individuals (UHR) is critical in prognosis and planning of potential personalised intervention strategies. Social and cognitive functioning observed in youth at UHR for psychosis may be protective against transition to clinically relevant illness. The current study used a computational method known as Spiking Neural Network (SNN) to identify the cognitive and social predictors of transitioning outcome. Participants (90 UHR, 81 Healthy Control (HC)) completed batteries of neuropsychological tests in the domains of verbal memory, working memory, processing speed, attention, executive function along with social skills-based performance at baseline and 4 × 6-month follow-up intervals. The UHR status was recorded as Remitters, Converters or Maintained. SNN were used to model interactions between variables across groups over time and classify UHR status. The performance of SNN was examined relative to other machine learning methods. Higher interaction between social and cognitive variables was seen for the Maintained, than Remitter subgroup. Findings identified the most important cognitive and social variables (particularly verbal memory, processing speed, attention, affect and interpersonal social functioning) that showed discriminative patterns in the SNN models of HC vs UHR subgroups, with accuracies up to 80%; outperforming other machine learning models (56-64% based on 18 months data). This finding is indicative of a promising direction for early detection of social and cognitive impairment in UHR individuals that may not anticipate transition to psychosis and implicate early initiated interventions to stem the impact of clinical symptoms of psychosis.

Constrained neuro fuzzy inference methodology for explainable personalised modelling with applications on gene expression data.

Interpretable machine learning models for gene expression datasets are important for understanding the decision-making process of a classifier and gaining insights on the underlying molecular processes of genetic conditions. Interpretable models can potentially support early diagnosis before full disease manifestation. This is particularly important yet, challenging for mental health. We hypothesise this is due to extreme heterogeneity issues which may be overcome and explained by personalised modelling techniques. Thus far, most machine learning methods applied to gene expression datasets, including deep neural networks, lack personalised interpretability. This paper proposes a new methodology named personalised constrained neuro fuzzy inference (PCNFI) for learning personalised rules from high dimensional datasets which are structurally and semantically interpretable. Case studies on two mental health related datasets (schizophrenia and bipolar disorders) have shown that the relatively short and simple personalised fuzzy rules provided enhanced interpretability as well as better classification performance compared to other commonly used machine learning methods. Performance test on a cancer dataset also showed that PCNFI matches previous benchmarks. Insights from our approach also indicated the importance of two genes (ATRX and TSPAN2) as possible biomarkers for early differentiation of ultra-high risk, bipolar and healthy individuals. These genes are linked to cognitive ability and impulsive behaviour. Our findings suggest a significant starting point for further research into the biological role of cognitive and impulsivity-related differences. With potential applications across bio-medical research, the proposed PCNFI method is promising for diagnosis, prognosis, and the design of personalised treatment plans for better outcomes in the future.

Prediction of Tinnitus Treatment Outcomes Based on EEG Sensors and TFI Score Using Deep Learning.

Tinnitus is a hearing disorder that is characterized by the perception of sounds in the absence of an external source. Currently, there is no pharmaceutical cure for tinnitus, however, multiple therapies and interventions have been developed that improve or control associated distress and anxiety. We propose a new Artificial Intelligence (AI) algorithm as a digital prognostic health system that models electroencephalographic (EEG) data in order to predict patients' responses to tinnitus therapies. The EEG data was collected from patients prior to treatment and 3-months following a sound-based therapy. Feature selection techniques were utilised to identify predictive EEG variables with the best accuracy. The patients' EEG features from both the frequency and functional connectivity domains were entered as inputs that carry knowledge extracted from EEG into AI algorithms for training and predicting therapy outcomes. The AI models differentiated the patients' outcomes into either therapy responder or non-responder, as defined by their Tinnitus Functional Index (TFI) scores, with accuracies ranging from 98%-100%. Our findings demonstrate the potential use of AI, including deep learning, for predicting therapy outcomes in tinnitus. The research suggests an optimal configuration of the EEG sensors that are involved in measuring brain functional changes in response to tinnitus treatments. It identified which EEG electrodes are the most informative sensors and how the EEG frequency and functional connectivity can better classify patients into the responder and non-responder groups. This has potential for real-time monitoring of patient therapy outcomes at home.

Personalised predictive modelling with brain-inspired spiking neural networks of longitudinal MRI neuroimaging data and the case study of dementia.

BACKGROUND: Longitudinal neuroimaging provides spatiotemporal brain data (STBD) measurement that can be utilised to understand dynamic changes in brain structure and/or function underpinning cognitive activities. Making sense of such highly interactive information is challenging, given that the features manifest intricate temporal, causal relations between the spatially distributed neural sources in the brain. METHODS: The current paper argues for the advancement of deep learning algorithms in brain-inspired spiking neural networks (SNN), capable of modelling structural data across time (longitudinal measurement) and space (anatomical components). The paper proposes a methodology and a computational architecture based on SNN for building personalised predictive models from longitudinal brain data to accurately detect, understand, and predict the dynamics of an individual's functional brain state. The methodology includes finding clusters of similar data to each individual, data interpolation, deep learning in a 3-dimensional brain-template structured SNN model, classification and prediction of individual outcome, visualisation of structural brain changes related to the predicted outcomes, interpretation of results, and individual and group predictive marker discovery. RESULTS: To demonstrate the functionality of the proposed methodology, the paper presents experimental results on a longitudinal magnetic resonance imaging (MRI) dataset derived from 175 older adults of the internationally recognised community-based cohort Sydney Memory and Ageing Study (MAS) spanning 6 years of follow-up. SIGNIFICANCE: The models were able to accurately classify and predict 2 years ahead of cognitive decline, such as mild cognitive impairment (MCI) and dementia with 95% and 91% accuracy, respectively. The proposed methodology also offers a 3-dimensional visualisation of the MRI models reflecting the dynamic patterns of regional changes in white matter hyperintensity (WMH) and brain volume over 6 years. CONCLUSION: The method is efficient for personalised predictive modelling on a wide range of neuroimaging longitudinal data, including also demographic, genetic, and clinical data. As a case study, it resulted in finding predictive markers for MCI and dementia as dynamic brain patterns using MRI data.

A State-of-Art Review of Digital Technologies for the Next Generation of Tinnitus Therapeutics.

Background: Digital processing has enabled the development of several generations of technology for tinnitus therapy. The first digital generation was comprised of digital Hearing Aids (HAs) and personal digital music players implementing already established sound-based therapies, as well as text based information on the internet. In the second generation Smart-phone applications (apps) alone or in conjunction with HAs resulted in more therapy options for users to select from. The 3rd generation of digital tinnitus technologies began with the emergence of many novel, largely neurophysiologically-inspired, treatment theories that drove development of processing; enabled through HAs, apps, the internet and stand-alone devices. We are now of the cusp of a 4th generation that will incorporate physiological sensors, multiple transducers and AI to personalize therapies. Aim: To review technologies that will enable the next generations of digital therapies for tinnitus. Methods: A "state-of-the-art" review was undertaken to answer the question: what digital technology could be applied to tinnitus therapy in the next 10 years? Google Scholar and PubMed were searched for the 10-year period 2011-2021. The search strategy used the following key words: "tinnitus" and ["HA," "personalized therapy," "AI" (and "methods" or "applications"), "Virtual reality," "Games," "Sensors" and "Transducers"], and "Hearables." Snowballing was used to expand the search from the identified papers. The results of the review were cataloged and organized into themes. Results: This paper identified digital technologies and research on the development of smart therapies for tinnitus. AI methods that could have tinnitus applications are identified and discussed. The potential of personalized treatments and the benefits of being able to gather data in ecologically valid settings are outlined. Conclusions: There is a huge scope for the application of digital technology to tinnitus therapy, but the uncertain mechanisms underpinning tinnitus present a challenge and many posited therapeutic approaches may not be successful. Personalized AI modeling based on biometric measures obtained through various sensor types, and assessments of individual psychology and lifestyles should result in the development of smart therapy platforms for tinnitus.

Deep Learning of Explainable EEG Patterns as Dynamic Spatiotemporal Clusters and Rules in a Brain-Inspired Spiking Neural Network.

The paper proposes a new method for deep learning and knowledge discovery in a brain-inspired Spiking Neural Networks (SNN) architecture that enhances the model's explainability while learning from streaming spatiotemporal brain data (STBD) in an incremental and on-line mode of operation. This led to the extraction of spatiotemporal rules from SNN models that explain why a certain decision (output prediction) was made by the model. During the learning process, the SNN created dynamic neural clusters, captured as polygons, which evolved in time and continuously changed their size and shape. The dynamic patterns of the clusters were quantitatively analyzed to identify the important STBD features that correspond to the most activated brain regions. We studied the trend of dynamically created clusters and their spike-driven events that occur together in specific space and time. The research contributes to: (1) enhanced interpretability of SNN learning behavior through dynamic neural clustering; (2) feature selection and enhanced accuracy of classification; (3) spatiotemporal rules to support model explainability; and (4) a better understanding of the dynamics in STBD in terms of feature interaction. The clustering method was applied to a case study of Electroencephalogram (EEG) data, recorded from a healthy control group (n = 21) and opiate use (n = 18) subjects while they were performing a cognitive task. The SNN models of EEG demonstrated different trends of dynamic clusters across the groups. This suggested to select a group of marker EEG features and resulted in an improved accuracy of EEG classification to 92%, when compared with all-feature classification. During learning of EEG data, the areas of neurons in the SNN model that form adjacent clusters (corresponding to neighboring EEG channels) were detected as fuzzy boundaries that explain overlapping activity of brain regions for each group of subjects.

Prediction of tinnitus masking benefit within a case series using a spiking neural network model.

Masking has been widely used as a tinnitus therapy, with large individual differences in its effectiveness. The basis of this variation is unknown. We examined individual tinnitus and psychological responses to three masking types, energetic masking (bilateral broadband static or rain noise [BBN]), informational masking (BBN with a notch at tinnitus pitch and 3-dimensional cues) and a masker combining both effects (BBN with spatial cues). Eleven participants with chronic tinnitus were followed for 12 months, each person used each masking approach for 3 months with a 1 month washout-baseline. The Tinnitus Functional Index (TFI), Tinnitus Rating Scales, Positive and Negative Affect Scale and Depression Anxiety Stress Scales, were measured every month of treatment. Electroencephalography (EEG) and psychoacoustic assessment was undertaken at baseline and following 3 months of each masking sound. The computational modeling of EEG data was based on the framework of brain-inspired Spiking Neural Network (SNN) architecture called NeuCube, designed for this study for mapping, learning, visualizing and classifying of brain activity patterns. EEG was related to clinically significant change in the TFI using the SNN model. The SNN framework was able to predict sound therapy responders (93% accuracy) from non-responders (100% accuracy) using baseline EEG recordings. The combination of energetic and informational masking was an effective treatment sound in more individuals than the other sounds used. Although the findings are promising, they are preliminary and require confirmation in independent and larger samples.

Prediction of Acoustic Residual Inhibition of Tinnitus Using a Brain-Inspired Spiking Neural Network Model.

Auditory Residual Inhibition (ARI) is a temporary suppression of tinnitus that occurs in some people following the presentation of masking sounds. Differences in neural response to ARI stimuli may enable classification of tinnitus and a tailored approach to intervention in the future. In an exploratory study, we investigated the use of a brain-inspired artificial neural network to examine the effects of ARI on electroencephalographic function, as well as the predictive ability of the model. Ten tinnitus patients underwent two auditory stimulation conditions (constant and amplitude modulated broadband noise) at two time points and were then characterised as responders or non-responders, based on whether they experienced ARI or not. Using a spiking neural network model, we evaluated concurrent neural patterns generated across space and time from features of electroencephalographic data, capturing the neural dynamic changes before and after stimulation. Results indicated that the model may be used to predict the effect of auditory stimulation on tinnitus on an individual basis. This approach may aid in the development of predictive models for treatment selection.

Interpretability of Spatiotemporal Dynamics of the Brain Processes Followed by Mindfulness Intervention in a Brain-Inspired Spiking Neural Network Architecture.

Mindfulness training is associated with improvements in psychological wellbeing and cognition, yet the specific underlying neurophysiological mechanisms underpinning these changes are uncertain. This study uses a novel brain-inspired artificial neural network to investigate the effect of mindfulness training on electroencephalographic function. Participants completed a 4-tone auditory oddball task (that included targets and physically similar distractors) at three assessment time points. In Group A (n = 10), these tasks were given immediately prior to 6-week mindfulness training, immediately after training and at a 3-week follow-up; in Group B (n = 10), these were during an intervention waitlist period (3 weeks prior to training), pre-mindfulness training and post-mindfulness training. Using a spiking neural network (SNN) model, we evaluated concurrent neural patterns generated across space and time from features of electroencephalographic data capturing the neural dynamics associated with the event-related potential (ERP). This technique capitalises on the temporal dynamics of the shifts in polarity throughout the ERP and spatially across electrodes. Findings support anteriorisation of connection weights in response to distractors relative to target stimuli. Right frontal connection weights to distractors were associated with trait mindfulness (positively) and depression (inversely). Moreover, mindfulness training was associated with an increase in connection weights to targets (bilateral frontal, left frontocentral, and temporal regions only) and distractors. SNN models were superior to other machine learning methods in the classification of brain states as a function of mindfulness training. Findings suggest SNN models can provide useful information that differentiates brain states based on distinct task demands and stimuli, as well as changes in brain states as a function of psychological intervention.

New avenue for the geriatric depression scale: Rasch transformation enhances reliability of assessment.

BACKGROUND: Depression is a common problem in older adults. The 15-item Geriatric Depression Scale (GDS-15) is a widely used psychometric tool for measuring depression in the elderly, but its psychometric properties have not been yet rigorously investigated. The aim was to evaluate psychometric properties of the GDS-15 and improve precision of the instrument by applying Rasch analysis and deriving conversion tables for transformation of raw scores into interval level data. METHODS: The data was extracted from the prospective cohort Sydney Memory and Ageing Study of initially not demented individuals aged 70 years and older. The GDS-15 items scores of 212 participants (47.2% males) were analysed using the dichotomous Rasch model. RESULTS: Initially poor reliability of the GDS-15, Person Separation Index (PSI) = 0.68, was improved by combining locally dependent items into seven super-items. These modifications improved reliability of the GDS-15 (PSI = 0.78) and resulted in the best Rasch model fit (χ2(28)=37.72, p = =0.104), strict unidimensionality and scale invariance across personal factors such as gender, diagnostic and language background. LIMITATIONS: Presence of participants with cognitive impairment may be a potential limitation. CONCLUSIONS: Reliability and psychometric characteristics of the GDS-15 were improved by minor modifications and now satisfy expectations of the unidimensional Rasch model. By using Rasch transformation tables published here psychiatrists, psychologists and researchers can transform GDS raw scores into interval-level data, which improves reliability of the GDS-15 without the need to modify its original response format. These findings increase accuracy of clinical psychometric assessments, leading to more precise diagnosis of depression in the elderly.

Personalised modelling with spiking neural networks integrating temporal and static information.

This paper proposes a new personalised prognostic/diagnostic system that supports classification, prediction and pattern recognition when both static and dynamic/spatiotemporal features are presented in a dataset. The system is based on a proposed clustering method (named d2WKNN) for optimal selection of neighbouring samples to an individual with respect to the integration of both static (vector-based) and temporal individual data. The most relevant samples to an individual are selected to train a Personalised Spiking Neural Network (PSNN) that learns from sets of streaming data to capture the space and time association patterns. The generated time-dependant patterns resulted in a higher accuracy of classification/prediction (80% to 93%) when compared with global modelling and conventional methods. In addition, the PSNN models can support interpretability by creating personalised profiling of an individual. This contributes to a better understanding of the interactions between features. Therefore, an end-user can comprehend what interactions in the model have led to a certain decision (outcome). The proposed PSNN model is an analytical tool, applicable to several real-life health applications, where different data domains describe a person's health condition. The system was applied to two case studies: (1) classification of spatiotemporal neuroimaging data for the investigation of individual response to treatment and (2) prediction of risk of stroke with respect to temporal environmental data. For both datasets, besides the temporal data, static health data were also available. The hyper-parameters of the proposed system, including the PSNN models and the d2WKNN clustering parameters, are optimised for each individual.

Spiking Neural Network Modelling Approach Reveals How Mindfulness Training Rewires the Brain.

There has been substantial interest in Mindfulness Training (MT) to understand how it can benefit healthy individuals as well as people with a broad range of health conditions. Research has begun to delineate associated changes in brain function. However, whether measures of brain function can be used to identify individuals who are more likely to respond to MT remains unclear. The present study applies a recently developed brain-inspired Spiking Neural Network (SNN) model to electroencephalography (EEG) data to provide novel insight into: i) brain function in depression; ii) the effect of MT on depressed and non-depressed individuals; and iii) neurobiological characteristics of depressed individuals who respond to mindfulness. Resting state EEG was recorded from before and after a 6 week MT programme in 18 participants. Based on self-report, 3 groups were formed: non-depressed (ND), depressed before but not after MT (responsive, D+) and depressed both before and after MT (unresponsive, D-). The proposed SNN, which utilises a standard brain-template, was used to model EEG data and assess connectivity, as indicated by activation levels across scalp regions (frontal, frontocentral, temporal, centroparietal and occipitoparietal), at baseline and follow-up. Results suggest an increase in activation following MT that was site-specific as a function of the group. Greater initial activation levels were seen in ND compared to depressed groups, and this difference was maintained at frontal and occipitoparietal regions following MT. At baseline, D+ had great activation than D-. Following MT, frontocentral and temporal activation reached ND levels in D+ but remained low in D-. Findings support the SNN approach in distinguishing brain states associated with depression and responsiveness to MT. The results also demonstrated that the SNN approach can be used to predict the effect of mindfulness on an individual basis before it is even applied.

Modelling Peri-Perceptual Brain Processes in a Deep Learning Spiking Neural Network Architecture.

Familiarity of marketing stimuli may affect consumer behaviour at a peri-perceptual processing level. The current study introduces a method for deep learning of electroencephalogram (EEG) data using a spiking neural network (SNN) approach that reveals the complexity of peri-perceptual processes of familiarity. The method is applied to data from 20 participants viewing familiar and unfamiliar logos. The results support the potential of SNN models as novel tools in the exploration of peri-perceptual mechanisms that respond differentially to familiar and unfamiliar stimuli. Specifically, the activation pattern of the time-locked response identified by the proposed SNN model at approximately 200 milliseconds post-stimulus suggests greater connectivity and more widespread dynamic spatio-temporal patterns for familiar than unfamiliar logos. The proposed SNN approach can be applied to study other peri-perceptual or perceptual brain processes in cognitive and computational neuroscience.

Mapping, Learning, Visualization, Classification, and Understanding of fMRI Data in the NeuCube Evolving Spatiotemporal Data Machine of Spiking Neural Networks.

This paper introduces a new methodology for dynamic learning, visualization, and classification of functional magnetic resonance imaging (fMRI) as spatiotemporal brain data. The method is based on an evolving spatiotemporal data machine of evolving spiking neural networks (SNNs) exemplified by the NeuCube architecture [1]. The method consists of several steps: mapping spatial coordinates of fMRI data into a 3-D SNN cube (SNNc) that represents a brain template; input data transformation into trains of spikes; deep, unsupervised learning in the 3-D SNNc of spatiotemporal patterns from data; supervised learning in an evolving SNN classifier; parameter optimization; and 3-D visualization and model interpretation. Two benchmark case study problems and data are used to illustrate the proposed methodology-fMRI data collected from subjects when reading affirmative or negative sentences and another one-on reading a sentence or seeing a picture. The learned connections in the SNNc represent dynamic spatiotemporal relationships derived from the fMRI data. They can reveal new information about the brain functions under different conditions. The proposed methodology allows for the first time to analyze dynamic functional and structural connectivity of a learned SNN model from fMRI data. This can be used for a better understanding of brain activities and also for online generation of appropriate neurofeedback to subjects for improved brain functions. For example, in this paper, tracing the 3-D SNN model connectivity enabled us for the first time to capture prominent brain functional pathways evoked in language comprehension. We found stronger spatiotemporal interaction between left dorsolateral prefrontal cortex and left temporal while reading a negated sentence. This observation is obviously distinguishable from the patterns generated by either reading affirmative sentences or seeing pictures. The proposed NeuCube-based methodology offers also a superior classification accuracy when compared with traditional AI and statistical methods. The created NeuCube-based models of fMRI data are directly and efficiently implementable on high performance and low energy consumption neuromorphic platforms for real-time applications.

Evolving spatio-temporal data machines based on the NeuCube neuromorphic framework: Design methodology and selected applications.

The paper describes a new type of evolving connectionist systems (ECOS) called evolving spatio-temporal data machines based on neuromorphic, brain-like information processing principles (eSTDM). These are multi-modular computer systems designed to deal with large and fast spatio/spectro temporal data using spiking neural networks (SNN) as major processing modules. ECOS and eSTDM in particular can learn incrementally from data streams, can include 'on the fly' new input variables, new output class labels or regression outputs, can continuously adapt their structure and functionality, can be visualised and interpreted for new knowledge discovery and for a better understanding of the data and the processes that generated it. eSTDM can be used for early event prediction due to the ability of the SNN to spike early, before whole input vectors (they were trained on) are presented. A framework for building eSTDM called NeuCube along with a design methodology for building eSTDM using this is presented. The implementation of this framework in MATLAB, Java, and PyNN (Python) is presented. The latter facilitates the use of neuromorphic hardware platforms to run the eSTDM. Selected examples are given of eSTDM for pattern recognition and early event prediction on EEG data, fMRI data, multisensory seismic data, ecological data, climate data, audio-visual data. Future directions are discussed, including extension of the NeuCube framework for building neurogenetic eSTDM and also new applications of eSTDM.