Energy-Efficient Neuromorphic Architectures for Nuclear Radiation Detection Applications.

A comprehensive analysis and simulation of two memristor-based neuromorphic architectures for nuclear radiation detection is presented. Both scalable architectures retrofit a locally competitive algorithm to solve overcomplete sparse approximation problems by harnessing memristor crossbar execution of vector-matrix multiplications. The proposed systems demonstrate excellent accuracy and throughput while consuming minimal energy for radionuclide detection. To ensure that the simulation results of our proposed hardware are realistic, the memristor parameters are chosen from our own fabricated memristor devices. Based on these results, we conclude that memristor-based computing is the preeminent technology for a radiation detection platform.

Shift-symmetric configurations in two-dimensional cellular automata: Irreversibility, insolvability, and enumeration.

The search for symmetry, as an unusual yet profoundly appealing phenomenon, and the origin of regular, repeating configuration patterns have long been a central focus of complexity science and physics. To better grasp and understand symmetry of configurations in decentralized toroidal architectures, we employ group-theoretic methods, which allow us to identify and enumerate these inputs, and argue about irreversible system behaviors with undesired effects on many computational problems. The concept of so-called "configuration shift-symmetry" is applied to two-dimensional cellular automata as an ideal model of computation. Regardless of the transition function, the results show the universal insolvability of crucial distributed tasks, such as leader election, pattern recognition, hashing, and encryption. By using compact enumeration formulas and bounding the number of shift-symmetric configurations for a given lattice size, we efficiently calculate the probability of a configuration being shift-symmetric for a uniform or density-uniform distribution. Further, we devise an algorithm detecting the presence of shift-symmetry in a configuration. Given the resource constraints, the enumeration and probability formulas can directly help to lower the minimal expected error and provide recommendations for system's size and initialization. Besides cellular automata, the shift-symmetry analysis can be used to study the nonlinear behavior in various synchronous rule-based systems that include inference engines, Boolean networks, neural networks, and systolic arrays.

Emergent criticality through adaptive information processing in boolean networks.

We study information processing in populations of boolean networks with evolving connectivity and systematically explore the interplay between the learning capability, robustness, the network topology, and the task complexity. We solve a long-standing open question and find computationally that, for large system sizes N, adaptive information processing drives the networks to a critical connectivity K(c)=2. For finite size networks, the connectivity approaches the critical value with a power law of the system size N. We show that network learning and generalization are optimized near criticality, given that the task complexity and the amount of information provided surpass threshold values. Both random and evolved networks exhibit maximal topological diversity near K(c). We hypothesize that this diversity supports efficient exploration and robustness of solutions. Also reflected in our observation is that the variance of the fitness values is maximal in critical network populations. Finally, we discuss implications of our results for determining the optimal topology of adaptive dynamical networks that solve computational tasks.

Revisiting the edge of chaos: Again?

Does biological computation happen at some sort of "edge of chaos", a dynamical regime somewhere between order and chaos? And if so, is this a fundamental principle that underlies self-organization, evolution, and complex natural and artificial systems that are subjected to adaptation? In this article, we will review the literature on the fundamental principles of computation in natural and artificial systems at the "edge of chaos". The term was coined by Norman Packard in the late 1980s. Since then, the concept of "adaptation to the edge of chaos" was demonstrated and investigated in many fields where both simple and complex systems receive some sort of feedback. Besides reviewing both historic and recent literature, we will also review critical voices of the concept.

Reconfigurable perovskite nickelate electronics for artificial intelligence.

Reconfigurable devices offer the ability to program electronic circuits on demand. In this work, we demonstrated on-demand creation of artificial neurons, synapses, and memory capacitors in post-fabricated perovskite NdNiO3 devices that can be simply reconfigured for a specific purpose by single-shot electric pulses. The sensitivity of electronic properties of perovskite nickelates to the local distribution of hydrogen ions enabled these results. With experimental data from our memory capacitors, simulation results of a reservoir computing framework showed excellent performance for tasks such as digit recognition and classification of electrocardiogram heartbeat activity. Using our reconfigurable artificial neurons and synapses, simulated dynamic networks outperformed static networks for incremental learning scenarios. The ability to fashion the building blocks of brain-inspired computers on demand opens up new directions in adaptive networks.

Fast and Accurate Sparse Coding of Visual Stimuli With a Simple, Ultralow-Energy Spiking Architecture.

Memristive crossbars have become a popular means for realizing unsupervised and supervised learning techniques. In previous neuromorphic architectures with leaky integrate-and-fire neurons, the crossbar itself has been separated from the neuron capacitors to preserve mathematical rigor. In this paper, we sought to design a simplified sparse coding circuit without this restriction, resulting in a fast circuit that approximated a sparse coding operation at a minimal loss in accuracy. We showed that connecting the neurons directly to the crossbar resulted in a more energy-efficient sparse coding architecture and alleviated the need to prenormalize receptive fields. This paper provides derivations for the design of such a network, named the simple spiking locally competitive algorithm, as well as CMOS designs and results on the CIFAR and MNIST data sets. Compared to a nonspiking, nonapproximate model which scored 33% on CIFAR-10 with a single-layer classifier, this hardware scored 32% accuracy. When used with a state-of-the-art deep learning classifier, the nonspiking model achieved 82% and our simplified, spiking model achieved 80% while compressing the input data by 92%. Compared to a previously proposed spiking model, our proposed hardware consumed 99% less energy to do the same work at 21 × the throughput. Accuracy held out with online learning to a write variance of 3%, suitable for the often reported 4-bit resolution required for neuromorphic algorithms, with offline learning to a write variance of 27%, and with read variance to 40%. The proposed architecture's excellent accuracy, throughput, and significantly lower energy usage demonstrate the utility of our innovations.

From membranes to systems: self-configuration and self-replication in membrane systems.

Membrane systems are purely abstract computational models afar inspired by biological cells, their membranes, and their biochemistry. The inherently parallel nature of membrane systems makes them obviously highly inefficient to execute on a sequential von Neumann computer architecture and in addition, programming a membrane system is often a painstakingly difficult undertaking. The main goal of this paper is to provide some key elements for bringing membrane systems from the abstract model closer to a genuine, novel, and unconventional in silico computer architecture. In particular, we will address the mechanisms of self-configuration and self-replication on a macroscopic level and will discuss some general issues related to genuine hardware realizations on the microscopic level.

Feedforward Chemical Neural Network: An In Silico Chemical System That Learns xor.

Inspired by natural biochemicals that perform complex information processing within living cells, we design and simulate a chemically implemented feedforward neural network, which learns by a novel chemical-reaction-based analogue of backpropagation. Our network is implemented in a simulated chemical system, where individual neurons are separated from each other by semipermeable cell-like membranes. Our compartmentalized, modular design allows a variety of network topologies to be constructed from the same building blocks. This brings us towards general-purpose, adaptive learning in chemico: wet machine learning in an embodied dynamical system.

Assembly of a 3D Cellular Computer Using Folded E-Blocks.

The assembly of integrated circuits in three dimensions (3D) provides a possible solution to address the ever-increasing demands of modern day electronic devices. It has been suggested that by using the third dimension, devices with high density, defect tolerance, short interconnects and small overall form factors could be created. However, apart from pseudo 3D architecture, such as monolithic integration, die, or wafer stacking, the creation of paradigms to integrate electronic low-complexity cellular building blocks in architecture that has tile space in all three dimensions has remained elusive. Here, we present software and hardware foundations for a truly 3D cellular computational devices that could be realized in practice. The computing architecture relies on the scalable, self-configurable and defect-tolerant cell matrix. The hardware is based on a scalable and manufacturable approach for 3D assembly using folded polyhedral electronic blocks (E-blocks). We created monomers, dimers and 2 × 2 × 2 assemblies of polyhedral E-blocks and verified the computational capabilities by implementing simple logic functions. We further show that 63.2% more compact 3D circuits can be obtained with our design automation tools compared to a 2D architecture. Our results provide a proof-of-concept for a scalable and manufacture-ready process for constructing massive-scale 3D computational devices.

Modeling and Experimental Demonstration of a Hopfield Network Analog-to-Digital Converter with Hybrid CMOS/Memristor Circuits.

The purpose of this work was to demonstrate the feasibility of building recurrent artificial neural networks with hybrid complementary metal oxide semiconductor (CMOS)/memristor circuits. To do so, we modeled a Hopfield network implementing an analog-to-digital converter (ADC) with up to 8 bits of precision. Major shortcomings affecting the ADC's precision, such as the non-ideal behavior of CMOS circuitry and the specific limitations of memristors, were investigated and an effective solution was proposed, capitalizing on the in-field programmability of memristors. The theoretical work was validated experimentally by demonstrating the successful operation of a 4-bit ADC circuit implemented with discrete Pt/TiO2- x /Pt memristors and CMOS integrated circuit components.

Bio-inspired computing tissues: towards machines that evolve, grow, and learn.

Biological inspiration in the design of computing machines could allow the creation of new machines with promising characteristics such as fault-tolerance, self-replication or cloning, reproduction, evolution, adaptation and learning, and growth. The aim of this paper is to introduce bio-inspired computing tissues that might constitute a key concept for the implementation of 'living' machines. We first present a general overview of bio-inspired systems and the POE model that classifies bio-inspired machines along three axes. The Embryonics project--inspired by some of the basic processes of molecular biology--is described by means of the BioWatch application, a fault-tolerant and self-repairable watch. The main characteristics of the Embryonics project are the multicellular organization, the cellular differentiation, and the self-repair capabilities. The BioWall is intended as a reconfigurable computing tissue, capable of interacting with its environment by means of a large number of touch-sensitive elements coupled with a color display. For illustrative purposes, a large-scale implementation of the BioWatch on the BioWall's computational tissue is presented. We conclude the paper with a description of bio-inspired computing tissues and POEtic machines.

Damage spreading in spatial and small-world random Boolean networks.

The study of the response of complex dynamical social, biological, or technological networks to external perturbations has numerous applications. Random Boolean networks (RBNs) are commonly used as a simple generic model for certain dynamics of complex systems. Traditionally, RBNs are interconnected randomly and without considering any spatial extension and arrangement of the links and nodes. However, most real-world networks are spatially extended and arranged with regular, power-law, small-world, or other nonrandom connections. Here we explore the RBN network topology between extreme local connections, random small-world, and pure random networks, and study the damage spreading with small perturbations. We find that spatially local connections change the scaling of the Hamming distance at very low connectivities (K ≪ 1) and that the critical connectivity of stability K(s) changes compared to random networks. At higher K, this scaling remains unchanged. We also show that the Hamming distance of spatially local networks scales with a power law as the system size N increases, but with a different exponent for local and small-world networks. The scaling arguments for small-world networks are obtained with respect to the system sizes and strength of spatially local connections. We further investigate the wiring cost of the networks. From an engineering perspective, our new findings provide the key design trade-offs between damage spreading (robustness), the network's wiring cost, and the network's communication characteristics.

Training an asymmetric signal perceptron through reinforcement in an artificial chemistry.

State-of-the-art biochemical systems for medical applications and chemical computing are application-specific and cannot be reprogrammed or trained once fabricated. The implementation of adaptive biochemical systems that would offer flexibility through programmability and autonomous adaptation faces major challenges because of the large number of required chemical species as well as the timing-sensitive feedback loops required for learning. In this paper, we begin addressing these challenges with a novel chemical perceptron that can solve all 14 linearly separable logic functions. The system performs asymmetric chemical arithmetic, learns through reinforcement and supports both Michaelis-Menten as well as mass-action kinetics. To enable cascading of the chemical perceptrons, we introduce thresholds that amplify the outputs. The simplicity of our model makes an actual wet implementation, in particular by DNA-strand displacement, possible.

Online learning in a chemical perceptron.

Autonomous learning implemented purely by means of a synthetic chemical system has not been previously realized. Learning promotes reusability and minimizes the system design to simple input-output specification. In this article we introduce a chemical perceptron, the first full-featured implementation of a perceptron in an artificial (simulated) chemistry. A perceptron is the simplest system capable of learning, inspired by the functioning of a biological neuron. Our artificial chemistry is deterministic and discrete-time, and follows Michaelis-Menten kinetics. We present two models, the weight-loop perceptron and the weight-race perceptron, which represent two possible strategies for a chemical implementation of linear integration and threshold. Both chemical perceptrons can successfully identify all 14 linearly separable two-input logic functions and maintain high robustness against rate-constant perturbations. We suggest that DNA strand displacement could, in principle, provide an implementation substrate for our model, allowing the chemical perceptron to perform reusable, programmable, and adaptable wet biochemical computing.

Computational capabilities of random automata networks for reservoir computing.

This paper underscores the conjecture that intrinsic computation is maximal in systems at the "edge of chaos". We study the relationship between dynamics and computational capability in random Boolean networks (RBN) for reservoir computing (RC). RC is a computational paradigm in which a trained readout layer interprets the dynamics of an excitable component (called the reservoir) that is perturbed by external input. The reservoir is often implemented as a homogeneous recurrent neural network, but there has been little investigation into the properties of reservoirs that are discrete and heterogeneous. Random Boolean networks are generic and heterogeneous dynamical systems and here we use them as the reservoir. A RBN is typically a closed system; to use it as a reservoir we extend it with an input layer. As a consequence of perturbation, the RBN does not necessarily fall into an attractor. Computational capability in RC arises from a tradeoff between separability and fading memory of inputs. We find the balance of these properties predictive of classification power and optimal at critical connectivity. These results are relevant to the construction of devices which exploit the intrinsic dynamics of complex heterogeneous systems, such as biomolecular substrates.

Nature-inspired interconnects for self-assembled large-scale network-on-chip designs.

Future nanoscale electronics built up from an Avogadro number of components need efficient, highly scalable, and robust means of communication in order to be competitive with traditional silicon approaches. In recent years, the networks-on-chip (NoC) paradigm emerged as a promising solution to interconnect challenges in silicon-based electronics. Current NoC architectures are either highly regular or fully customized, both of which represent implausible assumptions for emerging bottom-up self-assembled molecular electronics that are generally assumed to have a high degree of irregularity and imperfection. Here, we pragmatically and experimentally investigate important design tradeoffs and properties of an irregular, abstract, yet physically plausible three-dimensional (3D) small-world interconnect fabric that is inspired by modern network-on-chip paradigms. We vary the framework's key parameters, such as the connectivity, number of switch nodes, and distribution of long- versus short-range connections, and measure the network's relevant communication characteristics. We further explore the robustness against link failures and the ability and efficiency to solve a simple toy problem, the synchronization task. The results confirm that (1) computation in irregular assemblies is a promising and disruptive computing paradigm for self-assembled nanoscale electronics and (2) that 3D small-world interconnect fabrics with a power-law decaying distribution of shortcut lengths are physically plausible and have major advantages over local two-dimensional and 3D regular topologies.

Damage spreading and criticality in finite random dynamical networks.

We systematically study and compare damage spreading at the sparse percolation (SP) limit for random Boolean and threshold networks with perturbations that are independent of the network size N. This limit is relevant to information and damage propagation in many technological and natural networks. Using finite-size scaling, we identify a new characteristic connectivity Ks, at which the average number of damaged nodes d[over ], after a large number of dynamical updates, is independent of N. Based on marginal damage spreading, we determine the critical connectivity Kc(sparse)(N) for finite N at the SP limit and show that it systematically deviates from Kc, established by the annealed approximation, even for large system sizes. Our findings can potentially explain the results recently obtained for gene regulatory networks and have important implications for the evolution of dynamical networks that solve specific tasks.

Gaze following: why (not) learn it?

We propose a computational model of the emergence of gaze following skills in infant-caregiver interactions. The model is based on the idea that infants learn that monitoring their caregiver's direction of gaze allows them to predict the locations of interesting objects or events in their environment (Moore & Corkum, 1994). Elaborating on this theory, we demonstrate that a specific Basic Set of structures and mechanisms is sufficient for gaze following to emerge. This Basic Set includes the infant's perceptual skills and preferences, habituation and reward-driven learning, and a structured social environment featuring a caregiver who tends to look at things the infant will find interesting. We review evidence that all elements of the Basic Set are established well before the relevant gaze following skills emerge. We evaluate the model in a series of simulations and show that it can account for typical development. We also demonstrate that plausible alterations of model parameters, motivated by findings on two different developmental disorders - autism and Williams syndrome - produce delays or deficits in the emergence of gaze following. The model makes a number of testable predictions. In addition, it opens a new perspective for theorizing about cross-species differences in gaze following.

On Enigmas and Oracles: looking back to the future.

The Turing Day was held at the Swiss Federal Institute of Technology (EPFL) in Lausanne, Switzerland, on 28 June 2002.