Simulation platform for pattern recognition based on reservoir computing with memristor networks.

Memristive systems and devices are potentially available for implementing reservoir computing (RC) systems applied to pattern recognition. However, the computational ability of memristive RC systems depends on intertwined factors such as system architectures and physical properties of memristive elements, which complicates identifying the key factor for system performance. Here we develop a simulation platform for RC with memristor device networks, which enables testing different system designs for performance improvement. Numerical simulations show that the memristor-network-based RC systems can yield high computational performance comparable to that of state-of-the-art methods in three time series classification tasks. We demonstrate that the excellent and robust computation under device-to-device variability can be achieved by appropriately setting network structures, nonlinearity of memristors, and pre/post-processing, which increases the potential for reliable computation with unreliable component devices. Our results contribute to an establishment of a design guide for memristive reservoirs toward the realization of energy-efficient machine learning hardware.

Transfer-RLS method and transfer-FORCE learning for simple and fast training of reservoir computing models.

Reservoir computing is a machine learning framework derived from a special type of recurrent neural network. Following recent advances in physical reservoir computing, some reservoir computing devices are thought to be promising as energy-efficient machine learning hardware for real-time information processing. To realize efficient online learning with low-power reservoir computing devices, it is beneficial to develop fast convergence learning methods with simpler operations. This study proposes a training method located in the middle between the recursive least squares (RLS) method and the least mean squares (LMS) method, which are standard online learning methods for reservoir computing models. The RLS method converges fast but requires updates of a huge matrix called a gain matrix, whereas the LMS method does not use a gain matrix but converges very slow. On the other hand, the proposed method called a transfer-RLS method does not require updates of the gain matrix in the main-training phase by updating that in advance (i.e., in a pre-training phase). As a result, the transfer-RLS method can work with simpler operations than the original RLS method without sacrificing much convergence speed. We numerically and analytically show that the transfer-RLS method converges much faster than the LMS method. Furthermore, we show that a modified version of the transfer-RLS method (called transfer-FORCE learning) can be applied to the first-order reduced and controlled error (FORCE) learning for a reservoir computing model with a closed-loop, which is challenging to train.

Estimation and prediction of ellipsoidal molecular shapes in organic crystals based on ellipsoid packing.

Crystal structure prediction has been one of the fundamental and challenging problems in materials science. It is computationally exhaustive to identify molecular conformations and arrangements in organic molecular crystals due to complexity in intra- and inter-molecular interactions. From a geometrical viewpoint, specific types of organic crystal structures can be characterized by ellipsoid packing. In particular, we focus on aromatic systems which are important for organic semiconductor materials. In this study, we aim to estimate the ellipsoidal molecular shapes of such crystals and predict them from single molecular descriptors. First, we identify the molecular crystals with molecular centroid arrangements that correspond to affine transformations of four basic cubic lattices, through topological analysis of the dataset of crystalline polycyclic aromatic molecules. The novelty of our method is that the topological data analysis is applied to arrangements of molecular centroids intead of those of atoms. For each of the identified crystals, we estimate the intracrystalline molecular shape based on the ellipsoid packing assumption. Then, we show that the ellipsoidal shape can be predicted from single molecular descriptors using a machine learning method. The results suggest that topological characterization of molecular arrangements is useful for structure prediction of organic semiconductor materials.

Comparing catch-up vaccination programs based on analysis of 2012-13 rubella outbreak in Kawasaki City, Japan.

During the 2012-13 rubella outbreak in Japan, local governments implemented subsidy programs for catch-up vaccination to mitigate the rubella outbreak and prevent congenital rubella syndrome (CRS). In most local governments, to prevent CRS, eligible persons of the subsidy program were women who were planning to have a child and men who were partners of pregnant women. On the other hand, in Kawasaki City, unimmunized men aged 23-39 years were additionally included in the eligible persons, because they were included in an unimmunized men group resulting from the historical transition of the national routine vaccination in Japan. The number of rubella cases in the city decreased earlier than that in the whole Japan. First, in order to estimate the effect of the catch-up vaccination campaign in Kawasaki City on the epidemic outcome, we performed numerical simulations with a Susceptible-Vaccinated-Exposed-Infectious-Recovered (SVEIR) model incorporating real data. The result indicated that the catch-up vaccination campaign showed a beneficial impact on the early decay of the rubella cases. Second, we numerically compared several different implementation strategies of catch-up vaccinations under a fixed amount of total vaccinations. As a result, we found that early and intensive vaccinations are vital for significant reduction in the number of rubella cases and CRS occurrences. Our study suggests that mathematical models with epidemiological and social data can contribute to identifying the most effective vaccination strategy.

Network structure-based interventions on spatial spread of epidemics in metapopulation networks.

Mathematical modeling of epidemics is fundamental to understand the mechanism of the disease outbreak and provides helpful indications for effectiveness of interventions for policy makers. The metapopulation network model has been used in the analysis of epidemic dynamics by taking individual migration between patches into account. However, so far, most of the previous studies unrealistically assume that transmission rates within patches are the same, neglecting the nonuniformity of intervention measures in hindering epidemics. Here, based on the assumption that interventions deployed in a patch depend on its population size or economic level, which have shown a positive correlation with the patch's degree in networks, we propose a metapopulation network model to explore a network structure-based intervention strategy, aiming at understanding the interplay between intervention strategy and other factors including mobility patterns, initial population, as well as the network structure. Our results demonstrate that interventions to patches with different intensity are able to suppress the epidemic spreading in terms of both the epidemic threshold and the final epidemic size. Specifically, the intervention strategy targeting the patches with high degree is able to efficiently suppress epidemics. In addition, a detrimental effect is also observed depending on the interplay between the intervention measures and the initial population distribution. Our study opens a path for understanding epidemic dynamics and provides helpful insights into the implementation of countermeasures for the control of epidemics in reality.

Spatially Arranged Sparse Recurrent Neural Networks for Energy Efficient Associative Memory.

The development of hardware neural networks, including neuromorphic hardware, has been accelerated over the past few years. However, it is challenging to operate very large-scale neural networks with low-power hardware devices, partly due to signal transmissions through a massive number of interconnections. Our aim is to deal with the issue of communication cost from an algorithmic viewpoint and study learning algorithms for energy-efficient information processing. Here, we consider two approaches to finding spatially arranged sparse recurrent neural networks with the high cost-performance ratio for associative memory. In the first approach following classical methods, we focus on sparse modular network structures inspired by biological brain networks and examine their storage capacity under an iterative learning rule. We show that incorporating long-range intermodule connections into purely modular networks can enhance the cost-performance ratio. In the second approach, we formulate for the first time an optimization problem where the network sparsity is maximized under the constraints imposed by a pattern embedding condition. We show that there is a tradeoff between the interconnection cost and the computational performance in the optimized networks. We demonstrate that the optimized networks can achieve a better cost-performance ratio compared with those considered in the first approach. We show the effectiveness of the optimization approach mainly using binary patterns and apply it also to gray-scale image restoration. Our results suggest that the presented approaches are useful in seeking more sparse and less costly connectivity of neural networks for the enhancement of energy efficiency in hardware neural networks.

Intervention threshold for epidemic control in susceptible-infected-recovered metapopulation models.

Metapopulation epidemic models describe epidemic dynamics in networks of spatially distant patches connected via pathways for migration of individuals. In the present study, we deal with a susceptible-infected-recovered (SIR) metapopulation model where the epidemic process in each patch is represented by an SIR model and the mobility of individuals is assumed to be a homogeneous diffusion. We consider two types of patches including high-risk and low-risk ones under the assumption that a local patch is changed from a high-risk one to a low-risk one by an intervention. We theoretically analyze the intervention threshold which indicates the critical fraction of low-risk patches for preventing a global epidemic outbreak. We show that an intervention targeted to high-degree patches is more effective for epidemic control than a random intervention. The theoretical results are validated by Monte Carlo simulations for synthetic and realistic scale-free patch networks. The theoretical results also reveal that the intervention threshold depends on the human mobility network and the mobility rate. Our approach is useful for exploring better local interventions aimed at containment of epidemics.

Recent advances in physical reservoir computing: A review.

Reservoir computing is a computational framework suited for temporal/sequential data processing. It is derived from several recurrent neural network models, including echo state networks and liquid state machines. A reservoir computing system consists of a reservoir for mapping inputs into a high-dimensional space and a readout for pattern analysis from the high-dimensional states in the reservoir. The reservoir is fixed and only the readout is trained with a simple method such as linear regression and classification. Thus, the major advantage of reservoir computing compared to other recurrent neural networks is fast learning, resulting in low training cost. Another advantage is that the reservoir without adaptive updating is amenable to hardware implementation using a variety of physical systems, substrates, and devices. In fact, such physical reservoir computing has attracted increasing attention in diverse fields of research. The purpose of this review is to provide an overview of recent advances in physical reservoir computing by classifying them according to the type of the reservoir. We discuss the current issues and perspectives related to physical reservoir computing, in order to further expand its practical applications and develop next-generation machine learning systems.

Bifurcation analysis of a mathematical model of atopic dermatitis to determine patient-specific effects of treatments on dynamic phenotypes.

Atopic dermatitis (AD) is a common inflammatory skin disease, whose incidence is currently increasing worldwide. AD has a complex etiology, involving genetic, environmental, immunological, and epidermal factors, and its pathogenic mechanisms have not yet been fully elucidated. Identification of AD risk factors and systematic understanding of their interactions are required for exploring effective prevention and treatment strategies for AD. We recently developed a mathematical model for AD pathogenesis to clarify mechanisms underlying AD onset and progression. This model describes a dynamic interplay between skin barrier, immune regulation, and environmental stress, and reproduced four types of dynamic behaviour typically observed in AD patients in response to environmental triggers. Here, we analyse bifurcations of the model to identify mathematical conditions for the system to demonstrate transitions between different types of dynamic behaviour that reflect respective severity of AD symptoms. By mathematically modelling effects of topical application of antibiotics, emollients, corticosteroids, and their combinations with different application schedules and doses, bifurcation analysis allows us to mathematically evaluate effects of the treatments on improving AD symptoms in terms of the patients' dynamic behaviour. The mathematical method developed in this study can be used to explore and improve patient-specific personalised treatment strategies to control AD symptoms.

Smoothing effect for spatially distributed renewable resources and its impact on power grid robustness.

In this paper, we show that spatial correlation of renewable energy outputs greatly influences the robustness of the power grids against large fluctuations of the effective power. First, we evaluate the spatial correlation among renewable energy outputs. We find that the spatial correlation of renewable energy outputs depends on the locations, while the influence of the spatial correlation of renewable energy outputs on power grids is not well known. Thus, second, by employing the topology of the power grid in eastern Japan, we analyze the robustness of the power grid with spatial correlation of renewable energy outputs. The analysis is performed by using a realistic differential-algebraic equations model. The results show that the spatial correlation of the energy resources strongly degrades the robustness of the power grid. Our results suggest that we should consider the spatial correlation of the renewable energy outputs when estimating the stability of power grids.

Interplay between epidemic spread and information propagation on metapopulation networks.

The spread of an infectious disease has been widely found to evolve with the propagation of information. Many seminal works have demonstrated the impact of information propagation on the epidemic spreading, assuming that individuals are static and no mobility is involved. Inspired by the recent observation of diverse mobility patterns, we incorporate the information propagation into a metapopulation model based on the mobility patterns and contagion process, which significantly alters the epidemic threshold. In more details, we find that both the information efficiency and the mobility patterns have essential impacts on the epidemic spread. We obtain different scenarios leading to the mitigation of the outbreak by appropriately integrating the mobility patterns and the information efficiency as well. The inclusion of the impacts of the information propagation into the epidemiological model is expected to provide an support to public health implications for the suppression of epidemics.

Robustness and fragility in coupled oscillator networks under targeted attacks.

The dynamical tolerance of coupled oscillator networks against local failures is studied. As the fraction of failed oscillator nodes gradually increases, the mean oscillation amplitude in the entire network decreases and then suddenly vanishes at a critical fraction as a phase transition. This critical fraction, widely used as a measure of the network robustness, was analytically derived for random failures but not for targeted attacks so far. Here we derive the general formula for the critical fraction, which can be applied to both random failures and targeted attacks. We consider the effects of targeting oscillator nodes based on their degrees. First we deal with coupled identical oscillators with homogeneous edge weights. Then our theory is applied to networks with heterogeneous edge weights and to those with nonidentical oscillators. The analytical results are validated by numerical experiments. Our results reveal the key factors governing the robustness and fragility of oscillator networks.

Robustness of coupled oscillator networks with heterogeneous natural frequencies.

Robustness of coupled oscillator networks against local degradation of oscillators has been intensively studied in this decade. The oscillation behavior on the whole network is typically reduced with an increase in the fraction of degraded (inactive) oscillators. The critical fraction of inactive oscillators, at which a transition from an oscillatory to a quiescent state occurs, has been used as a measure for the network robustness. The larger (smaller) this measure is, the more robust (fragile) the oscillatory behavior on the network is. Most previous studies have used oscillators with identical natural frequencies, for which the oscillators are necessarily synchronized and thereby the analysis is simple. In contrast, we focus on the effect of heterogeneity in the natural frequencies on the network robustness. First, we analytically derive the robustness measure for the coupled oscillator models with heterogeneous natural frequencies under some conditions. Then, we show that increasing the heterogeneity in natural frequencies makes the network fragile. Moreover, we discuss the optimal parameter condition to maximize the network robustness.

Parameter Scaling for Epidemic Size in a Spatial Epidemic Model with Mobile Individuals.

In recent years, serious infectious diseases tend to transcend national borders and widely spread in a global scale. The incidence and prevalence of epidemics are highly influenced not only by pathogen-dependent disease characteristics such as the force of infection, the latent period, and the infectious period, but also by human mobility and contact patterns. However, the effect of heterogeneous mobility of individuals on epidemic outcomes is not fully understood. Here, we aim to elucidate how spatial mobility of individuals contributes to the final epidemic size in a spatial susceptible-exposed-infectious-recovered (SEIR) model with mobile individuals in a square lattice. After illustrating the interplay between the mobility parameters and the other parameters on the spatial epidemic spreading, we propose an index as a function of system parameters, which largely governs the final epidemic size. The main contribution of this study is to show that the proposed index is useful for estimating how parameter scaling affects the final epidemic size. To demonstrate the effectiveness of the proposed index, we show that there is a positive correlation between the proposed index computed with the real data of human airline travels and the actual number of positive incident cases of influenza B in the entire world, implying that the growing incidence of influenza B is attributed to increased human mobility.

Oscillation dynamics underlie functional switching of NF-κB for B-cell activation.

Transcription factor nuclear factor kappa B (NF-κB) shows cooperative switch-like activation followed by prolonged oscillatory nuclear translocation in response to extracellular stimuli. These dynamics are important for activation of the NF-κB transcriptional machinery, however, NF-κB activity regulated by coordinated actions of these dynamics has not been elucidated at the system level. Using a variety of B cells with artificially rewired NF-κB signaling networks, we show that oscillations and switch-like activation of NF-κB can be dissected and that, under some conditions, these two behaviors are separated upon antigen receptor activation. Comprehensive quantitative experiments and mathematical analysis showed that the functional role of switch activation in the NF-κB system is to overcome transient IKK (IκB kinase) activity to amplify nuclear translocation of NF-κB, thereby inducing the prolonged NF-κB oscillatory behavior necessary for target gene expression and B-cell activation.

Intermittent Androgen Suppression: Estimating Parameters for Individual Patients Based on Initial PSA Data in Response to Androgen Deprivation Therapy.

When a physician decides on a treatment and its schedule for a specific patient, information gained from prior patients and experience in the past is taken into account. A more objective way to make such treatment decisions based on actual data would be useful to the clinician. Although there are many mathematical models proposed for various diseases, so far there is no mathematical method that accomplishes optimization of the treatment schedule using the information gained from past patients or "rapid learning" technology. In an attempt to use this approach, we integrate the information gained from patients previously treated with intermittent androgen suppression (IAS) with that from a current patient by first fitting the time courses of clinical data observed from the previously treated patients, then constructing the prior information of the parameter values of the mathematical model, and finally, maximizing the posterior probability for the parameters of the current patient using the prior information. Although we used data from prostate cancer patients, the proposed method is general, and thus can be applied to other diseases once an appropriate mathematical model is established for that disease.

Robustness of oscillatory behavior in correlated networks.

Understanding network robustness against failures of network units is useful for preventing large-scale breakdowns and damages in real-world networked systems. The tolerance of networked systems whose functions are maintained by collective dynamical behavior of the network units has recently been analyzed in the framework called dynamical robustness of complex networks. The effect of network structure on the dynamical robustness has been examined with various types of network topology, but the role of network assortativity, or degree-degree correlations, is still unclear. Here we study the dynamical robustness of correlated (assortative and disassortative) networks consisting of diffusively coupled oscillators. Numerical analyses for the correlated networks with Poisson and power-law degree distributions show that network assortativity enhances the dynamical robustness of the oscillator networks but the impact of network disassortativity depends on the detailed network connectivity. Furthermore, we theoretically analyze the dynamical robustness of correlated bimodal networks with two-peak degree distributions and show the positive impact of the network assortativity.

Dynamical robustness of coupled heterogeneous oscillators.

We study tolerance of dynamic behavior in networks of coupled heterogeneous oscillators to deterioration of the individual oscillator components. As the deterioration proceeds with reduction in dynamic behavior of the oscillators, an order parameter evaluating the level of global oscillation decreases and then vanishes at a certain critical point. We present a method to analytically derive a general formula for this critical point and an approximate formula for the order parameter in the vicinity of the critical point in networks of coupled Stuart-Landau oscillators. Using the critical point as a measure for dynamical robustness of oscillator networks, we show that the more heterogeneous the oscillator components are, the more robust the oscillatory behavior of the network is to the component deterioration. This property is confirmed also in networks of Morris-Lecar neuron models coupled through electrical synapses. Our approach could provide a useful framework for theoretically understanding the role of population heterogeneity in robustness of biological networks.

Epidemic spread on interconnected metapopulation networks.

Numerous real-world networks have been observed to interact with each other, resulting in interconnected networks that exhibit diverse, nontrivial behavior with dynamical processes. Here we investigate epidemic spreading on interconnected networks at the level of metapopulation. Through a mean-field approximation for a metapopulation model, we find that both the interaction network topology and the mobility probabilities between subnetworks jointly influence the epidemic spread. Depending on the interaction between subnetworks, proper controls of mobility can efficiently mitigate epidemics, whereas an extremely biased mobility to one subnetwork will typically cause a severe outbreak and promote the epidemic spreading. Our analysis provides a basic framework for better understanding of epidemic behavior in related transportation systems as well as for better control of epidemics by guiding human mobility patterns.

Random and targeted interventions for epidemic control in metapopulation models.

In general, different countries and communities respond to epidemics in accordance with their own control plans and protocols. However, owing to global human migration and mobility, strategic planning for epidemic control measures through the collaboration of relevant public health administrations is gaining importance for mitigating and containing large-scale epidemics. Here, we present a framework to evaluate the effectiveness of random (non-strategic) and targeted (strategic) epidemic interventions for spatially separated patches in metapopulation models. For a random intervention, we analytically derive the critical fraction of patches that receive epidemic interventions, above which epidemics are successfully contained. The analysis shows that the heterogeneity of patch connectivity makes it difficult to contain epidemics under the random intervention. We demonstrate that, particularly in such heterogeneously connected networks, targeted interventions are considerably effective compared to the random intervention. Our framework is useful for identifying the target areas where epidemic control measures should be focused.