Theoretical Exploration of Flexible Transmitter Model.

Neural network models generally involve two important components, i.e., network architecture and neuron model. Although there are abundant studies about network architectures, only a few neuron models have been developed, such as the MP neuron model developed in 1943 and the spiking neuron model developed in the 1950s. Recently, a new bio-plausible neuron model, flexible transmitter (FT) model (Zhang and Zhou, 2021), has been proposed. It exhibits promising behaviors, particularly on temporal-spatial signals, even when simply embedded into the common feedforward network architecture. This article attempts to understand the properties of the FT network (FTNet) theoretically. Under mild assumptions, we show that: 1) FTNet is a universal approximator; 2) the approximation complexity of FTNet can be exponentially smaller than those of commonly used real-valued neural networks with feedforward/recurrent architectures and is of the same order in the worst case; and 3) any local minimum of FTNet is the global minimum, implying that it is possible to identify global minima by local search algorithms.

Neural Network Gaussian Processes by Increasing Depth.

Recent years have witnessed an increasing interest in the correspondence between infinitely wide networks and Gaussian processes. Despite the effectiveness and elegance of the current neural network Gaussian process theory, to the best of our knowledge, all the neural network Gaussian processes (NNGPs) are essentially induced by increasing width. However, in the era of deep learning, what concerns us more regarding a neural network is its depth as well as how depth impacts the behaviors of a network. Inspired by a width-depth symmetry consideration, we use a shortcut network to show that increasing the depth of a neural network can also give rise to a Gaussian process, which is a valuable addition to the existing theory and contributes to revealing the true picture of deep learning. Beyond the proposed Gaussian process by depth, we theoretically characterize its uniform tightness property and the smallest eigenvalue of the Gaussian process kernel. These characterizations can not only enhance our understanding of the proposed depth-induced Gaussian process but also pave the way for future applications. Lastly, we examine the performance of the proposed Gaussian process by regression experiments on two benchmark datasets.

Towards understanding theoretical advantages of complex-reaction networks.

Complex-valued neural networks have attracted increasing attention in recent years, while it remains open on the advantages of complex-valued neural networks in comparison with real-valued networks. This work takes one step on this direction by introducing the complex-reaction network with fully-connected feed-forward architecture. We prove the universal approximation property for complex-reaction networks, and show that a class of radial functions can be approximated by a complex-reaction network using the polynomial number of parameters, whereas real-valued networks need at least exponential parameters to reach the same approximation level. For empirical risk minimization, we study the landscape and convergence of complex gradient descents. Our theoretical result shows that the critical point set of complex-reaction networks is a proper subset of that of real-valued networks, which may show some insights on finding the optimal solutions more easily for complex-reaction networks.

Flexible Transmitter Network.

Current neural networks are mostly built on the MP model, which usually formulates the neuron as executing an activation function on the real-valued weighted aggregation of signals received from other neurons. This letter proposes the flexible transmitter (FT) model, a novel bio-plausible neuron model with flexible synaptic plasticity. The FT model employs a pair of parameters to model the neurotransmitters between neurons and puts up a neuron-exclusive variable to record the regulated neurotrophin density. Thus, the FT model can be formulated as a two-variable, two-valued function, taking the commonly used MP neuron model as its particular case. This modeling manner makes the FT model biologically more realistic and capable of handling complicated data, even spatiotemporal data. To exhibit its power and potential, we present the flexible transmitter network (FTNet), which is built on the most common fully connected feedforward architecture taking the FT model as the basic building block. FTNet allows gradient calculation and can be implemented by an improved backpropagation algorithm in the complex-valued domain. Experiments on a broad range of tasks show that FTNet has power and potential in processing spatiotemporal data. This study provides an alternative basic building block in neural networks and exhibits the feasibility of developing artificial neural networks with neuronal plasticity.

[Effect of electroacupuncture on proprioception in cynomolgus monkeys after unilateral anterior cruciate ligament injury].

OBJECTIVE: To study the effect of electroacupuncture on proprioception in cynomolgus monkeys after unilateral anterior cruciate ligament (ACL) injury. METHODS: Twenty-seven cynomolgus monkeys were randomized equally into 3 groups, namely unilateral ACL injury with electroacupuncture group, unilateral ACL injury model group and blank control group. One week after modeling, the monkeys in electroacupuncture group were treated daily for 12 weeks with electroacupuncture at the acupoints Wei Yang, Yin Gu, Xi Yangguan and Qu Quan. At 4, 8, and 12 weeks during the treatment, the changes in the somatosensory evoked potentials (SEPs) and motor nerve conduction velocity (MCV) of the nerves of the ACL were examined with neural electrophysiological method; the changes in the morphology and the total and variable number of the proprioceptors in the ACL were observed with gold chloride staining. RESULTS; In the mokeys in the model group, the incubation period of the SEPs and MCV on the injured side of the knee were significantly extended and the amplitudes were decreased with the passage of time (P<0.05). Intervention with electroacupuncture significantly reduced the incubation period and increased the amplitudes of the SEPs and MCV (P<0.05). The total number of the proprioceptors in the ACL was significantly decreased and the variable number of the proprioceptors in ACL was increased with time in the monkeys in the model group (P<0.05); electroacupuncture intervention obviously slowed the reduction rate of total number of the proprioceptors (P<0.05) but without affecting the variable number of the proprioceptors (P>0.05). CoONCLUSION: ACL injury causes attenuation of the proprioception on the injury side, and electroacupuncture intervention can produce a positive effect on the proprioception in cynomolgus monkeys.

Brain activation and inhibition after acupuncture at Taichong and Taixi: resting-state functional magnetic resonance imaging.

Acupuncture can induce changes in the brain. However, the majority of studies to date have focused on a single acupoint at a time. In the present study, we observed activity changes in the brains of healthy volunteers before and after acupuncture at Taichong (LR3) and Taixi (KI3) using resting-state functional magnetic resonance imaging. Fifteen healthy volunteers underwent resting-state functional magnetic resonance imaging of the brain 15 minutes before acupuncture, then received acupuncture at Taichong and Taixi using the nail-pressing needle insertion method, after which the needle was retained in place for 30 minutes. Fifteen minutes after withdrawal of the needle, the volunteers underwent a further session of resting-state functional magnetic resonance imaging, which revealed that the amplitude of low-frequency fluctuation, a measure of spontaneous neuronal activity, increased mainly in the cerebral occipital lobe and middle occipital gyrus (Brodmann area 18/19), inferior occipital gyrus (Brodmann area 18) and cuneus (Brodmann area 18), but decreased mainly in the gyrus rectus of the frontal lobe (Brodmann area 11), inferior frontal gyrus (Brodmann area 44) and the center of the posterior lobe of the cerebellum. The present findings indicate that acupuncture at Taichong and Taixi specifically promote blood flow and activation in the brain areas related to vision, emotion and cognition, and inhibit brain areas related to emotion, attention, phonological and semantic processing, and memory.