Neural waves and computation in a neural net model I: Convolutional hierarchies.

The computational resources of a neuromorphic network model introduced earlier are investigated in the context of such hierarchical systems as the mammalian visual cortex. It is argued that a form of ubiquitous spontaneous local convolution, driven by spontaneously arising wave-like activity-which itself promotes local Hebbian modulation-enables logical gate-like neural motifs to form into hierarchical feed-forward structures of the Hubel-Wiesel type. Extra-synaptic effects are shown to play a significant rôle in these processes. The type of logic that emerges is not Boolean, confirming and extending earlier findings on the logic of schizophrenia.

Neural waves and computation in a neural net model II: Data-like structures and the dynamics of episodic memory.

The computational resources of a neuromorphic network model introduced earlier were investigated in the first paper of this series. It was argued that a form of ubiquitous spontaneous local convolution enabled logical gate-like neural motifs to form into hierarchical feed-forward structures of the Hubel-Wiesel type. Here we investigate concomitant data-like structures and their dynamic rôle in memory formation, retrieval, and replay. The mechanisms give rise to the need for general inhibitory sculpting, and the simulation of the replay of episodic memories, well known in humans and recently observed in rats. Other consequences include explanations of such findings as the directional flows of neural waves in memory formation and retrieval, visual anomalies and memory deficits in schizophrenia, and the operation of GABA agonist drugs in suppressing episodic memories. We put forward the hypothesis that all neural logical operations and feature extractions are of the convolutional hierarchical type described here and in the earlier paper, and exemplified by the Hubel-Wiesel model of the visual cortex, but that in more general cases the precise geometric layering might be obscured and so far undetected.

The effect of mutational robustness on the evolvability of multicellular organisms and eukaryotic cells.

Canalization involves mutational robustness, the lack of phenotypic change as a result of genetic mutations. Given the large divergence in phenotype across species, understanding the relationship between high robustness and evolvability has been of interest to both theorists and experimentalists. Although canalization was originally proposed in the context of multicellular organisms, the effect of multicellularity and other classes of hierarchical organization on evolvability has not been considered by theoreticians. We address this issue using a Boolean population model with explicit representation of an environment in which individuals with explicit genotype and a hierarchical phenotype representing multicellularity evolve. Robustness is described by a single real number between zero and one which emerges from the genotype-phenotype map. We find that high robustness is favoured in constant environments, and lower robustness is favoured after environmental change. Multicellularity and hierarchical organization severely constrain robustness: peak evolvability occurs at an absolute level of robustness of about 0.99 compared with values of about 0.5 in a classical neutral network model. These constraints result in a sharp peak of evolvability in which the maximum is set by the fact that the fixation of adaptive mutations becomes more improbable as robustness decreases. When robustness is put under genetic control, robustness levels leading to maximum evolvability are selected for, but maximal relative fitness appears to require recombination.

Neural waves and short-term memory in a neural net model.

We show that recognizable neural waveforms are reproduced in the model described in previous work. In so doing, we reproduce close matches to certain observed, though filtered, EEG-like measurements in closed mathematical form, to good approximations. Such neural waves represent the responses of individual networks to external and endogenous inputs and are presumably the carriers of the information used to perform computations in actual brains, which are complexes of interconnected networks. Then, we apply these findings to a question arising in short-term memory processing in humans. Namely, we show how the anomalously small number of reliable retrievals from short-term memory found in certain trials of the Sternberg task is related to the relative frequencies of the neural waves involved. This finding justifies the hypothesis of phase-coding, which has been posited as an explanation of this effect.

Predicting cryptic ligand binding sites based on normal modes guided conformational sampling.

To greatly expand the druggable genome, fast and accurate predictions of cryptic sites for small molecules binding in target proteins are in high demand. In this study, we have developed a fast and simple conformational sampling scheme guided by normal modes solved from the coarse-grained elastic models followed by atomistic backbone refinement and side-chain repacking. Despite the observations of complex and diverse conformational changes associated with ligand binding, we found that simply sampling along each of the lowest 30 modes is near optimal for adequately restructuring cryptic sites so they can be detected by existing pocket finding programs like fpocket and concavity. We further trained machine-learning protocols to optimize the combination of the sampling-enhanced pocket scores with other dynamic and conservation scores, which only slightly improved the performance. As assessed based on a training set of 84 known cryptic sites and a test set of 14 proteins, our method achieved high accuracy of prediction (with area under the receiver operating characteristic curve >0.8) comparable to the CryptoSite server. Compared with CryptoSite and other methods based on extensive molecular dynamics simulation, our method is much faster (1-2 hours for an average-size protein) and simpler (using only pocket scores), so it is suitable for high-throughput processing of large datasets of protein structures at the genome scale.

Outcome prediction in aneurysmal subarachnoid hemorrhage: a comparison of machine learning methods and established clinico-radiological scores.

Reliable prediction of outcomes of aneurysmal subarachnoid hemorrhage (aSAH) based on factors available at patient admission may support responsible allocation of resources as well as treatment decisions. Radiographic and clinical scoring systems may help clinicians estimate disease severity, but their predictive value is limited, especially in devising treatment strategies. In this study, we aimed to examine whether a machine learning (ML) approach using variables available on admission may improve outcome prediction in aSAH compared to established scoring systems. Combined clinical and radiographic features as well as standard scores (Hunt & Hess, WFNS, BNI, Fisher, and VASOGRADE) available on patient admission were analyzed using a consecutive single-center database of patients that presented with aSAH (n = 388). Different ML models (seven algorithms including three types of traditional generalized linear models, as well as a tree bosting algorithm, a support vector machine classifier (SVMC), a Naive Bayes (NB) classifier, and a multilayer perceptron (MLP) artificial neural net) were trained for single features, scores, and combined features with a random split into training and test sets (4:1 ratio), ten-fold cross-validation, and 50 shuffles. For combined features, feature importance was calculated. There was no difference in performance between traditional and other ML applications using traditional clinico-radiographic features. Also, no relevant difference was identified between a combined set of clinico-radiological features available on admission (highest AUC 0.78, tree boosting) and the best performing clinical score GCS (highest AUC 0.76, tree boosting). GCS and age were the most important variables for the feature combination. In this cohort of patients with aSAH, the performance of functional outcome prediction by machine learning techniques was comparable to traditional methods and established clinical scores. Future work is necessary to examine input variables other than traditional clinico-radiographic features and to evaluate whether a higher performance for outcome prediction in aSAH can be achieved.

Mask-Guided Convolutional Neural Network for Breast Tumor Prognostic Outcome Prediction on 3D DCE-MR Images.

In this proof-of-concept work, we have developed a 3D-CNN architecture that is guided by the tumor mask for classifying several patient-outcomes in breast cancer from the respective 3D dynamic contrast-enhanced MRI (DCE-MRI) images. The tumor masks on DCE-MRI images were generated using pre- and post-contrast images and validated by experienced radiologists. We show that our proposed mask-guided classification has a higher accuracy than that from either the full image without tumor masks (including background) or the masked voxels only. We have used two patient outcomes for this study: (1) recurrence of cancer after 5 years of imaging and (2) HER2 status, for comparing accuracies of different models. By looking at the activation maps, we conclude that an image-based prediction model using 3D-CNN could be improved by even a conservatively generated mask, rather than overly trusting an unguided, blind 3D-CNN. A blind CNN may classify accurately enough, while its attention may really be focused on a remote region within 3D images. On the other hand, only using a conservatively segmented region may not be as good for classification as using full images but forcing the model's attention toward the known regions of interest.

Microwave Imaging by Deep Learning Network: Feasibility and Training Method.

Microwave image reconstruction based on a deep-learning method is investigated in this paper. The neural network is capable of converting measured microwave signals acquired from a 24×24 antenna array at 4 GHz into a 128×128 image. To reduce the training difficulty, we first developed an autoencoder by which high-resolution images (128×128) were represented with 256×1 vectors; then we developed the second neural network which aimed to map microwave signals to the compressed features (256×1 vector). Two neural networks can be combined to a full network to make reconstructions, when both are successfully developed. The present two-stage training method reduces the difficulty in training deep learning networks (DLN) for inverse reconstruction. The developed neural network is validated by simulation examples and experimental data with objects in different shapes/sizes, placed in different locations, and with dielectric constant ranging from 2~6. Comparisons between the imaging results achieved by the present method and two conventional approaches: distorted Born iterative method (DBIM) and phase confocal method (PCM) are also provided.

Theory of cortical function.

Most models of sensory processing in the brain have a feedforward architecture in which each stage comprises simple linear filtering operations and nonlinearities. Models of this form have been used to explain a wide range of neurophysiological and psychophysical data, and many recent successes in artificial intelligence (with deep convolutional neural nets) are based on this architecture. However, neocortex is not a feedforward architecture. This paper proposes a first step toward an alternative computational framework in which neural activity in each brain area depends on a combination of feedforward drive (bottom-up from the previous processing stage), feedback drive (top-down context from the next stage), and prior drive (expectation). The relative contributions of feedforward drive, feedback drive, and prior drive are controlled by a handful of state parameters, which I hypothesize correspond to neuromodulators and oscillatory activity. In some states, neural responses are dominated by the feedforward drive and the theory is identical to a conventional feedforward model, thereby preserving all of the desirable features of those models. In other states, the theory is a generative model that constructs a sensory representation from an abstract representation, like memory recall. In still other states, the theory combines prior expectation with sensory input, explores different possible perceptual interpretations of ambiguous sensory inputs, and predicts forward in time. The theory, therefore, offers an empirically testable framework for understanding how the cortex accomplishes inference, exploration, and prediction.

Evaluating classification accuracy for modern learning approaches.

Deep learning neural network models such as multilayer perceptron (MLP) and convolutional neural network (CNN) are novel and attractive artificial intelligence computing tools. However, evaluation of the performance of these methods is not readily available for practitioners yet. We provide a tutorial for evaluating classification accuracy for various state-of-the-art learning approaches, including familiar shallow and deep learning methods. For qualitative response variables with more than two categories, many traditional accuracy measures such as sensitivity, specificity, and area under the receiver operating characteristic curve are not applicable and we have to consider their extensions properly. In this paper, a few important statistical concepts for multicategory classification accuracy are reviewed and their utilities for various learning algorithms are demonstrated with real medical examples. We offer problem-based R code to illustrate how to perform these statistical computations step by step. We expect that such analysis tools will become more familiar to practitioners and receive broader applications in biostatistics.

Road Extraction from Unmanned Aerial Vehicle Remote Sensing Images Based on Improved Neural Networks.

Roads are vital components of infrastructure, the extraction of which has become a topic of significant interest in the field of remote sensing. Because deep learning has been a popular method in image processing and information extraction, researchers have paid more attention to extracting road using neural networks. This article proposes the improvement of neural networks to extract roads from Unmanned Aerial Vehicle (UAV) remote sensing images. D-Linknet was first considered for its high performance; however, the huge scale of the net reduced computational efficiency. With a focus on the low computational efficiency problem of the popular D-LinkNet, this article made some improvements: (1) Replace the initial block with a stem block. (2) Rebuild the entire network based on ResNet units with a new structure, allowing for the construction of an improved neural network D-Linknetplus. (3) Add a 1 × 1 convolution layer before DBlock to reduce the input feature maps, reducing parameters and improving computational efficiency. Add another 1 × 1 convolution layer after DBlock to recover the required number of output channels. Accordingly, another improved neural network B-D-LinknetPlus was built. Comparisons were performed between the neural nets, and the verification were made with the Massachusetts Roads Dataset. The results show improved neural networks are helpful in reducing the network size and developing the precision needed for road extraction.

Alpha, delta and theta rhythms in a neural net model. Comparison with MEG data.

The aim of this study is to provide information regarding the comparison of a neural model to MEG measurements. Our study population consisted of 10 epileptic patients and 10 normal subjects. The epileptic patients had high MEG amplitudes characterized with θ (4-7 Hz) or δ (2-3 Hz) rhythms and absence of α-rhythm (8-13 Hz). The statistical analysis of such activities corresponded to Poisson distribution. Conversely, the MEG from normal subjects had low amplitudes, higher frequencies and presence of α-rhythm (8-13 Hz). Such activities were not synchronized and their distributions were Gauss. These findings were in agreement with our theoretical neural model. The comparison of the neural network with MEG data provides information about the status of brain function in epileptic and normal states.

Using machine learning to predict selenium and cadmium contents in rice grains from black shale-distributed farmland area.

Cadmium (Cd) and selenium (Se) are widely enriched in soil at black shale outcropping areas, with Cd levels exceeding the standard (2.0 mg/kg in 5.5 < pH ≤ 6.5) commonly. The prevention of Cd hazards and the safe development of Se-rich land resources are key issues that need to be urgently addressed. To ensure safe utilization of Se-rich land in the CdSe coexisting areas, 158 rice samples, their corresponding rhizosphere soils, and 8069 topsoil samples were collected and tested in the paddy fields of Ankang City, Shaanxi Province, where black shales are widely exposed. The results showed that 43 % of the topsoil samples were Se-rich soil (Se > 0.4 mg/kg) wherein 79 % and 3 % of Cd concentrations exceeded the screening value and control value, respectively, according to the GB15618-2018 standard. Meanwhile, 63 % of the rice samples were Se rich (Se > 0.04 mg/kg) and the Cd content exceeded the prescribed limit (0.2 mg/kg) in Se-rich rice by 26 %. There was no significant positive correlation between the Se and Cd contents in the rice grains and the Se and Cd contents in the corresponding rhizosphere soil. The factors influencing Se and Cd uptake in rice were SiO2, CaO, P, S, pH, and TFe2O3. Accordingly, an artificial neural network (ANN) and multiple linear regression model (MLR) were used to predict Cd and Se bioaccumulation in rice grains. The stability and accuracy of the ANN model were better than those of the MLR model. Based on survey data and the prediction results of the ANN model, a safe planting zoning of Se-rich rice was proposed, which provided a reference for the scientific planning of land resources.

Visuo-dynamic self-modelling of soft robotic systems.

Soft robots exhibit complex nonlinear dynamics with large degrees of freedom, making their modelling and control challenging. Typically, reduced-order models in time or space are used in addressing these challenges, but the resulting simplification limits soft robot control accuracy and restricts their range of motion. In this work, we introduce an end-to-end learning-based approach for fully dynamic modelling of any general robotic system that does not rely on predefined structures, learning dynamic models of the robot directly in the visual space. The generated models possess identical dimensionality to the observation space, resulting in models whose complexity is determined by the sensory system without explicitly decomposing the problem. To validate the effectiveness of our proposed method, we apply it to a fully soft robotic manipulator, and we demonstrate its applicability in controller development through an open-loop optimization-based controller. We achieve a wide range of dynamic control tasks including shape control, trajectory tracking and obstacle avoidance using a model derived from just 90 min of real-world data. Our work thus far provides the most comprehensive strategy for controlling a general soft robotic system, without constraints on the shape, properties, or dimensionality of the system.

Parameter Inference for an Astrocyte Model using Machine Learning Approaches.

Astrocytes are the largest subset of glial cells and perform structural, metabolic, and regulatory functions. They are directly involved in the communication at neuronal synapses and the maintenance of brain homeostasis. Several disorders, such as Alzheimer's, epilepsy, and schizophrenia, have been associated with astrocyte dysfunction. Computational models on various spatial levels have been proposed to aid in the understanding and research of astrocytes. The difficulty of computational astrocyte models is to fastly and precisely infer parameters. Physics informed neural networks (PINNs) use the underlying physics to infer parameters and, if necessary, dynamics that can not be observed. We have applied PINNs to estimate parameters for a computational model of an astrocytic compartment. The addition of two techniques helped with the gradient pathologies of the PINNS, the dynamic weighting of various loss components and the addition of Transformers. To overcome the issue that the neural network only learned the time dependence but did not know about eventual changes of the input stimulation to the astrocyte model, we followed an adaptation of PINNs from control theory (PINCs). In the end, we were able to infer parameters from artificial, noisy data, with stable results for the computational astrocyte model.

Learning about reward identities and time.

We discuss three empirical findings that we think any theory attempting to integrate interval timing with associative learning concepts will need to address. These empirical phenomena all come from studies that combine peak timing procedures with reinforcer devaluation or conditional discrimination tasks commonly employed, respectively, in interval timing or associative learning research traditions. The three phenomena we discuss include: (1) the observation that disruptions in reward identity encoding have little to no impact on the encoding of reward time in the peak procedure (Delamateret al., 2018), (2) the findings that organisms tend to average their time estimates when presented with a stimulus compound consisting of separately learned stimuli indicating short or long reward times but that such temporal averaging, itself, is sensitive to post-conditioning selective reward devaluation, and (3) that rats can learn a temporal patterning task in which two stimuli presented independently indicate one time to reward availability while their compound indicates another. We review our prior results and present new findings illustrating these three phenomena and we discuss the special challenges they pose for cascade theories of timing, for multiple-oscillator models, and for any approach that attempts to integrate interval timing and associative models. We close by illustrating some ways in which multi-layer connectionist network models might begin to address some of our key findings. We believe this will require an approach that includes separate mechanisms that code for reward identity and time, but that does so in a way that permits for integration between the two systems.

Convolutional Neural Network-Based ECG-Assisted Diagnosis for Coal Workers.

OBJECTIVE: To process and extract electrocardiogram (ECG, ECG, or EKG) features using a convolutional neural network (CNN) to establish an ECG-assisted diagnosis model. METHODS: Coal workers who underwent physical examinations at Gequan Mine Hospital and Dongpang Mine Hospital of Hebei Jizhong Energy from July 2020 to September 2020 were selected as the study subjects. The ECG images were preprocessed. We use Python software and convolutional neural network to establish ECG images recognition and classification model.We usecalibration curve, calibration-in-the-large, Brier score, specificity, sensitivity, F1 score, Kappa value, accuracy, and area under the curve (AUC) of ROC to evaluate the performance of the model. RESULTS: The number of abnormal ECG results was 849, and the rate of abnormal results was 25.02%. The test set accuracies of the sinus bradycardia model, nonspecific intraventricular conduction delay model, myocardial ischemia model, and sinus tachycardia model were 97.66%, 96.49%, 93.62%, and 93.02%, respectively; sensitivities were 96.63%, 96.30%, 96.88% and 95.24%, respectively; specificities were 98.78%, 96.67%, 86.67%, and 90.90%, respectively; Brier scores were 0.03, 0.07, 0.09, and 0.11, respectively; Calibration-in-the-large values were 0.026, 0.110, 0.041, and 0.098, respectively. CONCLUSIONS: The convolutional neural network model can accurately identify the main ECG abnormality types of coal workers. Additionally, the main ECG abnormalities in these coal company workers were sinus bradycardia, non-specific intraventricular conduction delay, myocardial ischemia, and sinus tachycardia.

Ultrafast simulation of large-scale neocortical microcircuitry with biophysically realistic neurons.

Understanding the activity of the mammalian brain requires an integrative knowledge of circuits at distinct scales, ranging from ion channel gating to circuit connectomics. Computational models are regularly employed to understand how multiple parameters contribute synergistically to circuit behavior. However, traditional models of anatomically and biophysically realistic neurons are computationally demanding, especially when scaled to model local circuits. To overcome this limitation, we trained several artificial neural network (ANN) architectures to model the activity of realistic multicompartmental cortical neurons. We identified an ANN architecture that accurately predicted subthreshold activity and action potential firing. The ANN could correctly generalize to previously unobserved synaptic input, including in models containing nonlinear dendritic properties. When scaled, processing times were orders of magnitude faster compared with traditional approaches, allowing for rapid parameter-space mapping in a circuit model of Rett syndrome. Thus, we present a novel ANN approach allowing for rapid, detailed network experiments using inexpensive and commonly available computational resources.

A faster way to model neuronal circuitry.

Artificial neural networks could pave the way for efficiently simulating large-scale models of neuronal networks in the nervous system.

What kind of network is the brain?

The different areas of the cerebral cortex are linked by a network of white matter, comprising the myelinated axons of pyramidal cells. Is this network a neural net, in the sense that representations of the world are embodied in the structure of the net, its pattern of nodes, and connections? Or is it a communications network, where the same physical substrate carries different information from moment to moment? This question is part of the larger question of whether the brain is better modeled by connectionism or by symbolic artificial intelligence (AI), but we review it in the specific context of the psychophysics of stimulus comparison and the format and protocol of information transmission over the long-range tracts of the brain.