Individual differences in nucleus accumbens dopamine receptors predict development of addiction-like behavior: a computational approach.

Clinical and experimental observations show individual differences in the development of addiction. Increasing evidence supports the hypothesis that dopamine receptor availability in the nucleus accumbens (NAc) predisposes drug reinforcement. Here, modeling striatal-midbrain dopaminergic circuit, we propose a reinforcement learning model for addiction based on the actor-critic model of striatum. Modeling dopamine receptors in the NAc as modulators of learning rate for appetitive--but not aversive--stimuli in the critic--but not the actor--we define vulnerability to addiction as a relatively lower learning rate for the appetitive stimuli, compared to aversive stimuli, in the critic. We hypothesize that an imbalance in this learning parameter used by appetitive and aversive learning systems can result in addiction. We elucidate that the interaction between the degree of individual vulnerability and the duration of exposure to drug has two progressive consequences: deterioration of the imbalance and establishment of an abnormal habitual response in the actor. Using computational language, the proposed model describes how development of compulsive behavior can be a function of both degree of drug exposure and individual vulnerability. Moreover, the model describes how involvement of the dorsal striatum in addiction can be augmented progressively. The model also interprets other forms of addiction, such as obesity and pathological gambling, in a common mechanism with drug addiction. Finally, the model provides an answer for the question of why behavioral addictions are triggered in Parkinson's disease patients by D2 dopamine agonist treatments.

Neuromuscular control of the point to point and oscillatory movements of a sagittal arm with the actor-critic reinforcement learning method.

In this study, we have used a single link system with a pair of muscles that are excited with alpha and gamma signals to achieve both point to point and oscillatory movements with variable amplitude and frequency.The system is highly nonlinear in all its physical and physiological attributes. The major physiological characteristics of this system are simultaneous activation of a pair of nonlinear muscle-like-actuators for control purposes, existence of nonlinear spindle-like sensors and Golgi tendon organ-like sensor, actions of gravity and external loading. Transmission delays are included in the afferent and efferent neural paths to account for a more accurate representation of the reflex loops.A reinforcement learning method with an actor-critic (AC) architecture instead of middle and low level of central nervous system (CNS), is used to track a desired trajectory. The actor in this structure is a two layer feedforward neural network and the critic is a model of the cerebellum. The critic is trained by state-action-reward-state-action (SARSA) method. The critic will train the actor by supervisory learning based on the prior experiences. Simulation studies of oscillatory movements based on the proposed algorithm demonstrate excellent tracking capability and after 280 epochs the RMS error for position and velocity profiles were 0.02, 0.04 rad and rad/s, respectively.

Automatic segmentation of thalamus from brain MRI integrating fuzzy clustering and dynamic contours.

Thalamus is an important neuro-anatomic structure in the brain. In this paper, an automated method is presented to segment thalamus from magnetic resonance images (MRI). The method is based on a discrete dynamic contour model that consists of vertices and edges connecting adjacent vertices. The model starts from an initial contour and deforms by external and internal forces. Internal forces are calculated from local geometry of the model and external forces are estimated from desired image features such as edges. However, thalamus has low contrast and discontinues edges on MRI, making external force estimation a challenge. The problem is solved using a new algorithm based on fuzzy C-means (FCM) unsupervised clustering, Prewitt edge-finding filter, and morphological operators. In addition, manual definition of the initial contour for the model makes the final segmentation operator-dependent. To eliminate this dependency, new methods are developed for generating the initial contour automatically. The proposed approaches are evaluated and validated by comparing automatic and radiologist's segmentation results and illustrating their agreement.

Automatic landmark extraction from image data using modified growing neural gas network.

A new method for automatic landmark extraction from MR brain images is presented. In this method, landmark extraction is accomplished by modifying growing neural gas (GNG), which is a neural-network-based cluster-seeking algorithm. Using modified GNG (MGNG) corresponding dominant points of contours extracted from two corresponding images are found. These contours are borders of segmented anatomical regions from brain images. The presented method is compared to: 1) the node splitting-merging Kohonen model and 2) the Teh-Chin algorithm (a well-known approach for dominant points extraction of ordered curves). It is shown that the proposed algorithm has lower distortion error, ability of extracting landmarks from two corresponding curves simultaneously, and also generates the best match according to five medical experts.

Identification and real-time position control of a servo-hydraulic rotary actuator by means of a neurobiologically motivated algorithm.

This paper presents a new intelligent approach for adaptive control of a nonlinear dynamic system. A modified version of the brain emotional learning based intelligent controller (BELBIC), a bio-inspired algorithm based upon a computational model of emotional learning which occurs in the amygdala, is utilized for position controlling a real laboratorial rotary electro-hydraulic servo (EHS) system. EHS systems are known to be nonlinear and non-smooth due to many factors such as leakage, friction, hysteresis, null shift, saturation, dead zone, and especially fluid flow expression through the servo valve. The large value of these factors can easily influence the control performance in the presence of a poor design. In this paper, a mathematical model of the EHS system is derived, and then the parameters of the model are identified using the recursive least squares method. In the next step, a BELBIC is designed based on this dynamic model and utilized to control the real laboratorial EHS system. To prove the effectiveness of the modified BELBIC's online learning ability in reducing the overall tracking error, results have been compared to those obtained from an optimal PID controller, an auto-tuned fuzzy PI controller (ATFPIC), and a neural network predictive controller (NNPC) under similar circumstances. The results demonstrate not only excellent improvement in control action, but also less energy consumption.

Relaxed fault-tolerant hardware implementation of neural networks in the presence of multiple transient errors.

Reliability should be identified as the most important challenge in future nano-scale very large scale integration (VLSI) implementation technologies for the development of complex integrated systems. Normally, fault tolerance (FT) in a conventional system is achieved by increasing its redundancy, which also implies higher implementation costs and lower performance that sometimes makes it even infeasible. In contrast to custom approaches, a new class of applications is categorized in this paper, which is inherently capable of absorbing some degrees of vulnerability and providing FT based on their natural properties. Neural networks are good indicators of imprecision-tolerant applications. We have also proposed a new class of FT techniques called relaxed fault-tolerant (RFT) techniques which are developed for VLSI implementation of imprecision-tolerant applications. The main advantage of RFT techniques with respect to traditional FT solutions is that they exploit inherent FT of different applications to reduce their implementation costs while improving their performance. To show the applicability as well as the efficiency of the RFT method, the experimental results for implementation of a face-recognition computationally intensive neural network and its corresponding RFT realization are presented in this paper. The results demonstrate promising higher performance of artificial neural network VLSI solutions for complex applications in faulty nano-scale implementation environments.

Prediction of breeding values for dairy cattle using artificial neural networks and neuro-fuzzy systems.

Developing machine learning and soft computing techniques has provided many opportunities for researchers to establish new analytical methods in different areas of science. The objective of this study is to investigate the potential of two types of intelligent learning methods, artificial neural networks and neuro-fuzzy systems, in order to estimate breeding values (EBV) of Iranian dairy cattle. Initially, the breeding values of lactating Holstein cows for milk and fat yield were estimated using conventional best linear unbiased prediction (BLUP) with an animal model. Once that was established, a multilayer perceptron was used to build ANN to predict breeding values from the performance data of selection candidates. Subsequently, fuzzy logic was used to form an NFS, a hybrid intelligent system that was implemented via a local linear model tree algorithm. For milk yield the correlations between EBV and EBV predicted by the ANN and NFS were 0.92 and 0.93, respectively. Corresponding correlations for fat yield were 0.93 and 0.93, respectively. Correlations between multitrait predictions of EBVs for milk and fat yield when predicted simultaneously by ANN were 0.93 and 0.93, respectively, whereas corresponding correlations with reference EBV for multitrait NFS were 0.94 and 0.95, respectively, for milk and fat production.

Estimating the mutual information between bilateral breast in thermograms using nonparametric windows.

Comparison between contra lateral breast images is one of the effective methods in breast cancer detection. Asymmetric temperature distribution can be an indicator of abnormality. The mutual information is a good measure of nonlinear correlation. It is a measure that captures linear and nonlinear dependencies, without requiring the specification of any kind of model of dependence. Therefore, it is suitable for our abnormality indicator. Although nonparametric windows is a numerically expensive technique but it is accurate. The reason is that nonparametric windows incorporate an interpolation model which enhances the resolution to a highly oversampled image. For our purposes we worked with sixty simulated breast thermal images. It is shown that the more similar the thermal image of right breast to the thermal image of left breast, the closer the normalized mutual information value to one.

A neurocomputational model for cocaine addiction.

Based on the dopamine hypotheses of cocaine addiction and the assumption of decrement of brain reward system sensitivity after long-term drug exposure, we propose a computational model for cocaine addiction. Utilizing average reward temporal difference reinforcement learning, we incorporate the elevation of basal reward threshold after long-term drug exposure into the model of drug addiction proposed by Redish. Our model is consistent with the animal models of drug seeking under punishment. In the case of nondrug reward, the model explains increased impulsivity after long-term drug exposure. Furthermore, the existence of a blocking effect for cocaine is predicted by our model.