A Supervised Learning Algorithm for Multilayer Spiking Neural Networks Based on Temporal Coding Toward Energy-Efficient VLSI Processor Design.

Spiking neural networks (SNNs) are brain-inspired mathematical models with the ability to process information in the form of spikes. SNNs are expected to provide not only new machine-learning algorithms but also energy-efficient computational models when implemented in very-large-scale integration (VLSI) circuits. In this article, we propose a novel supervised learning algorithm for SNNs based on temporal coding. A spiking neuron in this algorithm is designed to facilitate analog VLSI implementations with analog resistive memory, by which ultrahigh energy efficiency can be achieved. We also propose several techniques to improve the performance on recognition tasks and show that the classification accuracy of the proposed algorithm is as high as that of the state-of-the-art temporal coding SNN algorithms on the MNIST and Fashion-MNIST datasets. Finally, we discuss the robustness of the proposed SNNs against variations that arise from the device manufacturing process and are unavoidable in analog VLSI implementation. We also propose a technique to suppress the effects of variations in the manufacturing process on the recognition performance.

Early dynamics of chronic myeloid leukemia on nilotinib predicts deep molecular response.

Chronic myeloid leukemia (CML) is a myeloproliferative disorder caused by the BCR-ABL1 tyrosine kinase. Although ABL1-specific tyrosine kinase inhibitors (TKIs) including nilotinib have dramatically improved the prognosis of patients with CML, the TKI efficacy depends on the individual patient. In this work, we found that the patients with different nilotinib responses can be classified by using the estimated parameters of our simple dynamical model with two common laboratory findings. Furthermore, our proposed method identified patients who failed to achieve a treatment goal with high fidelity according to the data collected only at three initial time points during nilotinib therapy. Since our model relies on the general properties of TKI response, our framework would be applicable to CML patients who receive frontline nilotinib or other TKIs.

Model-size reduction for reservoir computing by concatenating internal states through time.

Reservoir computing (RC) is a machine learning algorithm that can learn complex time series from data very rapidly based on the use of high-dimensional dynamical systems, such as random networks of neurons, called "reservoirs." To implement RC in edge computing, it is highly important to reduce the amount of computational resources that RC requires. In this study, we propose methods that reduce the size of the reservoir by inputting the past or drifting states of the reservoir to the output layer at the current time step. To elucidate the mechanism of model-size reduction, the proposed methods are analyzed based on information processing capacity proposed by Dambre et al. (Sci Rep 2:514, 2012). In addition, we evaluate the effectiveness of the proposed methods on time-series prediction tasks: the generalized Hénon-map and NARMA. On these tasks, we found that the proposed methods were able to reduce the size of the reservoir up to one tenth without a substantial increase in regression error.

Personalizing Androgen Suppression for Prostate Cancer Using Mathematical Modeling.

Using a dataset of 150 patients treated with intermittent androgen suppression (IAS) through a fixed treatment schedule, we retrospectively designed a personalized treatment schedule mathematically for each patient. We estimated 100 sets of parameter values for each patient by randomly resampling each patient's time points to take into account the uncertainty for observations of prostate specific antigen (PSA). Then, we identified 3 types and classified patients accordingly: in type (i), the relapse, namely the divergence of PSA, can be prevented by IAS; in type (ii), the relapse can be delayed by IAS later than by continuous androgen suppression (CAS); in type (iii) IAS was not beneficial and therefore CAS would have been more appropriate in the long run. Moreover, we obtained a treatment schedule of hormone therapy by minimizing the PSA of 3 years later in the worst case scenario among the 100 parameter sets by searching exhaustively all over the possible treatment schedules. If the most frequent type among 100 sets was type (i), the maximal PSA tended to be kept less than 100 ng/ml longer in IAS than in CAS, while there was no statistical difference for the other cases. Thus, mathematically personalized IAS should be studied prospectively.

Intermittent Androgen Suppression: Estimating Parameters for Individual Patients Based on Initial PSA Data in Response to Androgen Deprivation Therapy.

When a physician decides on a treatment and its schedule for a specific patient, information gained from prior patients and experience in the past is taken into account. A more objective way to make such treatment decisions based on actual data would be useful to the clinician. Although there are many mathematical models proposed for various diseases, so far there is no mathematical method that accomplishes optimization of the treatment schedule using the information gained from past patients or "rapid learning" technology. In an attempt to use this approach, we integrate the information gained from patients previously treated with intermittent androgen suppression (IAS) with that from a current patient by first fitting the time courses of clinical data observed from the previously treated patients, then constructing the prior information of the parameter values of the mathematical model, and finally, maximizing the posterior probability for the parameters of the current patient using the prior information. Although we used data from prostate cancer patients, the proposed method is general, and thus can be applied to other diseases once an appropriate mathematical model is established for that disease.

Predicting disease progression from short biomarker series using expert advice algorithm.

Well-trained clinicians may be able to provide diagnosis and prognosis from very short biomarker series using information and experience gained from previous patients. Although mathematical methods can potentially help clinicians to predict the progression of diseases, there is no method so far that estimates the patient state from very short time-series of a biomarker for making diagnosis and/or prognosis by employing the information of previous patients. Here, we propose a mathematical framework for integrating other patients' datasets to infer and predict the state of the disease in the current patient based on their short history. We extend a machine-learning framework of "prediction with expert advice" to deal with unstable dynamics. We construct this mathematical framework by combining expert advice with a mathematical model of prostate cancer. Our model predicted well the individual biomarker series of patients with prostate cancer that are used as clinical samples.

Robustness of oscillatory behavior in correlated networks.

Understanding network robustness against failures of network units is useful for preventing large-scale breakdowns and damages in real-world networked systems. The tolerance of networked systems whose functions are maintained by collective dynamical behavior of the network units has recently been analyzed in the framework called dynamical robustness of complex networks. The effect of network structure on the dynamical robustness has been examined with various types of network topology, but the role of network assortativity, or degree-degree correlations, is still unclear. Here we study the dynamical robustness of correlated (assortative and disassortative) networks consisting of diffusively coupled oscillators. Numerical analyses for the correlated networks with Poisson and power-law degree distributions show that network assortativity enhances the dynamical robustness of the oscillator networks but the impact of network disassortativity depends on the detailed network connectivity. Furthermore, we theoretically analyze the dynamical robustness of correlated bimodal networks with two-peak degree distributions and show the positive impact of the network assortativity.

Dynamical robustness of coupled heterogeneous oscillators.

We study tolerance of dynamic behavior in networks of coupled heterogeneous oscillators to deterioration of the individual oscillator components. As the deterioration proceeds with reduction in dynamic behavior of the oscillators, an order parameter evaluating the level of global oscillation decreases and then vanishes at a certain critical point. We present a method to analytically derive a general formula for this critical point and an approximate formula for the order parameter in the vicinity of the critical point in networks of coupled Stuart-Landau oscillators. Using the critical point as a measure for dynamical robustness of oscillator networks, we show that the more heterogeneous the oscillator components are, the more robust the oscillatory behavior of the network is to the component deterioration. This property is confirmed also in networks of Morris-Lecar neuron models coupled through electrical synapses. Our approach could provide a useful framework for theoretically understanding the role of population heterogeneity in robustness of biological networks.

Efficient recovery of dynamic behavior in coupled oscillator networks.

We study an effective method to recover dynamic activity in coupled oscillator networks that have been damaged and lost oscillatory dynamics owing to some inactivated or deteriorated oscillator elements. Recovery of the dynamic behavior can be achieved by newly connecting intact oscillators to the network. We analytically and numerically examine the proportion of the oscillators that are needed to be supported by intact oscillators for recovery of oscillation dynamics. Our results show that it can be more effective to preferentially support active oscillators in the damaged network than to preferentially support inactivated ones. The conditions for this counterintuitive result are discussed. Our framework could be a theoretical foundation for understanding regeneration of oscillatory dynamics in physical and biological systems.

Dynamical robustness in complex networks: the crucial role of low-degree nodes.

Many social, biological, and technological networks consist of a small number of highly connected components (hubs) and a very large number of loosely connected components (low-degree nodes). It has been commonly recognized that such heterogeneously connected networks are extremely vulnerable to the failure of hubs in terms of structural robustness of complex networks. However, little is known about dynamical robustness, which refers to the ability of a network to maintain its dynamical activity against local perturbations. Here we demonstrate that, in contrast to the structural fragility, the nonlinear dynamics of heterogeneously connected networks can be highly vulnerable to the failure of low-degree nodes. The crucial role of low-degree nodes results from dynamical processes where normal (active) units compensate for the failure of neighboring (inactive) units at the expense of a reduction in their own activity. Our finding highlights the significant difference between structural and dynamical robustness in complex networks.

Robustness of multilayer oscillator networks.

We consider the robustness of multilayer networks composed of active and inactive oscillators from the viewpoint of interlayer coupling effects through the aging transition [H. Daido and K. Nakanishi, Phys. Rev. Lett. 93, 104101 (2004)]. We show in detail that two-layer networks increase or decrease their robustness depending on interlayer coupling schemes compared with single-layer networks. In addition, we find that an increase of mismatches of oscillator types (active or inactive) among interlayer-connected oscillators reduces the robustness of the networks with mean-field, chain, and diffusive interlayer couplings in two-layer networks. Moreover, we discuss the robustness of networks with more than two layers.