RESUMEN
Catastrophic forgetting is a phenomenon in which a neural network, upon learning a new task, struggles to maintain its performance on previously learned tasks. It is a common challenge in the realm of continual learning (CL) through neural networks. The mammalian brain addresses catastrophic forgetting by consolidating memories in different parts of the brain, involving the hippocampus and the neocortex. Taking inspiration from this brain strategy, we present a CL framework that combines a plastic model simulating the fast learning capabilities of the hippocampus and a stable model representing the slow consolidation nature of the neocortex. To supplement this, we introduce a variational autoencoder (VAE)-based pseudo memory for rehearsal purposes. In addition by applying lateral inhibition masks on the gradients of the convolutional layer, we aim at damping the activity of adjacent neurons and introduce a sleep phase to reorganize the learned representations. Empirical evaluation demonstrates the positive impact of such additions on the performance of our proposed framework; we evaluate the proposed model on several class-incremental and domain-incremental datasets and compare it with the standard benchmark algorithms, showing significant improvements. With the aim to showcase practical applicability, we implement the algorithm in a physical environment for object classification using a soft pneumatic gripper. The algorithm learns new classes incrementally in real time and also exhibits significant backward knowledge transfer (KT).
RESUMEN
This Perspective presents the Modular-Integrative Modeling approach, a novel framework in neuroscience for developing brain models that blend biological realism with functional performance to provide a holistic view on brain function in interaction with the body and environment.
RESUMEN
Considering the variability and heterogeneity of motor impairment in children with Movement Disorders (MDs), the assessment of postural control becomes essential. For its assessment, only a few tools objectively quantify and recognize the difference among children with MDs. In this study, we use the Virtual Reality Rehabilitation System (VRRS) for assessing the postural control in children with MD. Furthermore, 16 children (mean age 10.68 ± 3.62 years, range 4.29-18.22 years) were tested with VRRS by using a stabilometric balance platform. Postural parameters, related to the movements of the Centre of Pressure (COP), were collected and analyzed. Three different MD groups were identified according to the prevalent MD: dystonia, chorea and chorea-dystonia. Statistical analyses tested the differences among MD groups in the VRRS-derived COP variables. The mean distance, root mean square, excursion, velocity and frequency values of the dystonia group showed significant differences (p < 0.05) between the chorea group and the chorea-dystonia group. Technology provides quantitative data to support clinical assessment: in this case, the VRRS detected differences among the MD patterns, identifying specific group features. This tool could be useful also for monitoring the longitudinal trajectories and detecting post-treatment changes.
RESUMEN
This paper presents Soft DAgger, an efficient imitation learning-based approach for training control solutions for soft robots. To demonstrate the effectiveness of the proposed algorithm, we implement it on a two-module soft robotic arm involved in the task of writing letters in 3D space. Soft DAgger uses a dynamic behavioral map of the soft robot, which maps the robot's task space to its actuation space. The map acts as a teacher and is responsible for predicting the optimal actions for the soft robot based on its previous state action history, expert demonstrations, and current position. This algorithm achieves generalization ability without depending on costly exploration techniques or reinforcement learning-based synthetic agents. We propose two variants of the control algorithm and demonstrate that good generalization capabilities and improved task reproducibility can be achieved, along with a consistent decrease in the optimization time and samples. Overall, Soft DAgger provides a practical control solution to perform complex tasks in fewer samples with soft robots. To the best of our knowledge, our study is an initial exploration of imitation learning with online optimization for soft robot control.
RESUMEN
The sense of touch plays a fundamental role in enabling us to interact with our surrounding environment. Indeed, the presence of tactile feedback in prostheses greatly assists amputees in doing daily tasks. In this line, the present study proposes an integration of artificial tactile and proprioception receptors for texture discrimination under varying scanning speeds. Here, we fabricated a soft biomimetic fingertip including an 8 × 8 array tactile sensor and a piezoelectric sensor to mimic Merkel, Meissner, and Pacinian mechanoreceptors in glabrous skin, respectively. A hydro-elastomer sensor was fabricated as an artificial proprioception sensor (muscle spindles) to assess the instantaneous speed of the biomimetic fingertip. In this study, we investigated the concept of the complex receptive field of RA-I and SA-I afferents for naturalistic textures. Next, to evaluate the synergy between the mechanoreceptors and muscle spindle afferents, ten naturalistic textures were manipulated by a soft biomimetic fingertip at six different speeds. The sensors' outputs were converted into neuromorphic spike trains to mimic the firing pattern of biological mechanoreceptors. These spike responses are then analyzed using machine learning classifiers and neural coding paradigms to explore the multi-sensory integration in real experiments. This synergy between muscle spindle and mechanoreceptors in the proposed neuromorphic system represents a generalized texture discrimination scheme and interestingly irrespective of the scanning speed.
Asunto(s)
Percepción del Tacto , Tacto , Tacto/fisiología , Mecanorreceptores/fisiología , Piel , PropiocepciónRESUMEN
BACKGROUND: Attention-Deficit/Hyperactivity Disorder (ADHD) is a highly heterogeneous diagnostic category, encompassing several endophenotypes and comorbidities, including sleep problems. However, no predictor of clinical long-term trajectories or comorbidity has yet been established. Sleep EEG has been proposed as a potential tool for evaluating the synaptic strength during development, as well as the cortical thickness, which is presumed to be altered in ADHD. We investigated whether the slope of the Slow Waves (SWs), a microstructural parameter of the sleep EEG, was a potential predictive parameter for psychiatric comorbidities and neuropsychological dimensions in ADHD. METHODS: 70 children (58 m; 8.76 ± 2.77 y) with ADHD who underwent psychiatric and neurologic evaluations and a standard EEG recording during naps were investigated. After sleep EEG analysis, we grouped the extracted SWs in bins of equal amplitude and then measured the associations, through generalized linear regression, between their maximum downward slopes (MDS) and the individual scores obtained from clinical rating scales. RESULTS: The presence of Multiple Anxiety Disorders was positively associated with MDS of medium amplitude SWs in temporo-posterior left areas. The Child Behavior Checklist scores showed negative associations in the same areas for small SWs. The presence of autistic traits was positively associated with MDS of high amplitude SWs in bilateral anterior and temporal left areas. The WISC-IV Processing Speed Index showed negative associations with MDS of small-to-medium SWs in anterior and temporal right areas, while positive associations in posterior and temporal left areas. CONCLUSIONS: Consistency of association clusters' localization on the scalp suggests that variations in the local MDS, revealing alterations of local synaptic strength and/or in daytime use of certain cortical circuits, could underlie specific neurodevelopmental trajectories resulting in different ADHD clinical phenotypes.
Asunto(s)
Trastorno por Déficit de Atención con Hiperactividad , Humanos , SueñoRESUMEN
Saccadic eye-movements play a crucial role in visuo-motor control by allowing rapid foveation onto new targets. However, the neural processes governing saccades adaptation are not fully understood. Saccades, due to the short-time of execution (20-100 ms) and the absence of sensory information for online feedback control, must be controlled in a ballistic manner. Incomplete measurements of the movement trajectory, such as the visual endpoint error, are supposedly used to form internal predictions about the movement kinematics resulting in predictive control. In order to characterize the synaptic and neural circuit mechanisms underlying predictive saccadic control, we have reconstructed the saccadic system in a digital controller embedding a spiking neural network of the cerebellum with spike timing-dependent plasticity (STDP) rules driving parallel fiber-Purkinje cell long-term potentiation and depression (LTP and LTD). This model implements a control policy based on a dual plasticity mechanism, resulting in the identification of the roles of LTP and LTD in regulating the overall quality of saccade kinematics: it turns out that LTD increases the accuracy by decreasing visual error and LTP increases the peak speed. The control policy also required cerebellar PCs to be divided into two subpopulations, characterized by burst or pause responses. To our knowledge, this is the first model that explains in mechanistic terms the visual error and peak speed regulation of ballistic eye movements in forward mode exploiting spike-timing to regulate firing in different populations of the neuronal network. This elementary model of saccades could be extended and applied to other more complex cases in which single jerks are concatenated to compose articulated and coordinated movements.
Asunto(s)
Células de Purkinje , Movimientos Sacádicos , Cerebelo/fisiología , Movimientos Oculares , Plasticidad Neuronal/fisiología , Células de Purkinje/fisiologíaRESUMEN
Conflictual cues and unexpected changes in human real-case scenarios may be detrimental to the execution of tasks by artificial agents, thus affecting their performance. Meta-learning applied to reinforcement learning may enhance the design of control algorithms, where an outer learning system progressively adjusts the operation of an inner learning system, leading to practical benefits for the learning schema. Here, we developed a brain-inspired meta-learning framework for inhibition cognitive control that i) exploits the meta-learning principles in the neuromodulation theory proposed by Doya, ii) relies on a well-established neural architecture that contains distributed learning systems in the human brain, and iii) proposes optimization rules of meta-learning hyperparameters that mimic the dynamics of the major neurotransmitters in the brain. We tested an artificial agent in inhibiting the action command in two well-known tasks described in the literature: NoGo and Stop-Signal Paradigms. After a short learning phase, the artificial agent learned to react to the hold signal, and hence to successfully inhibit the motor command in both tasks, via the continuous adjustment of the learning hyperparameters. We found a significant increase in global accuracy, right inhibition, and a reduction in the latency time required to cancel the action process, i.e., the Stop-signal reaction time. We also performed a sensitivity analysis to evaluate the behavioral effects of the meta-parameters, focusing on the serotoninergic modulation of the dopamine release. We demonstrated that brain-inspired principles can be integrated into artificial agents to achieve more flexible behavior when conflictual inhibitory signals are present in the environment.
Asunto(s)
Dopamina , Refuerzo en Psicología , Encéfalo , Cognición , Dopamina/fisiología , Humanos , Aprendizaje/fisiologíaRESUMEN
Parkinson's disease (PD) is known to affect the brain motor circuits involving the basal ganglia (BG) and to induce, among other signs, general slowness and paucity of movements. In upper limb movements, PD patients show a systematic prolongation of movement duration while maintaining a sufficient level of endpoint accuracy. PD appears to cause impairments not only in movement execution, but also in movement initiation and planning, as revealed by abnormal preparatory activity of motor-related brain areas. Grasping movement is affected as well, particularly in the coordination of the hand aperture with the transport phase. In the last fifty years, numerous behavioral studies attempted to clarify the mechanisms underlying these anomalies, speculating on the plausible role that the BG-thalamo-cortical circuitry may play in normal and pathological motor control. Still, many questions remain open, especially concerning the management of the speed-accuracy tradeoff and the online feedback control. In this review, we summarize the literature results on reaching and grasping in parkinsonian patients. We analyze the relevant hypotheses on the origins of dysfunction, by focusing on the motor control aspects involved in the different movement phases and the corresponding role played by the BG. We conclude with an insight into the innovative stimulation techniques and computational models recently proposed, which might be helpful in further clarifying the mechanisms through which PD affects reaching and grasping movements.
Asunto(s)
Corteza Motora , Enfermedad de Parkinson , Ganglios Basales , Mano , Humanos , Movimiento/fisiología , Desempeño Psicomotor/fisiologíaRESUMEN
Recent studies have identified rotational dynamics in motor cortex (MC), which many assume arise from intrinsic connections in MC. However, behavioral and neurophysiological studies suggest that MC behaves like a feedback controller where continuous sensory feedback and interactions with other brain areas contribute substantially to MC processing. We investigated these apparently conflicting theories by building recurrent neural networks that controlled a model arm and received sensory feedback from the limb. Networks were trained to counteract perturbations to the limb and to reach toward spatial targets. Network activities and sensory feedback signals to the network exhibited rotational structure even when the recurrent connections were removed. Furthermore, neural recordings in monkeys performing similar tasks also exhibited rotational structure not only in MC but also in somatosensory cortex. Our results argue that rotational structure may also reflect dynamics throughout the voluntary motor system involved in online control of motor actions.
Asunto(s)
Retroalimentación Sensorial/fisiología , Macaca mulatta/fisiología , Corteza Motora/fisiología , Corteza Somatosensorial/fisiología , Animales , Modelos NeurológicosRESUMEN
Touch and pain sensations are complementary aspects of daily life that convey crucial information about the environment while also providing protection to our body. Technological advancements in prosthesis design and control mechanisms assist amputees to regain lost function but often they have no meaningful tactile feedback or perception. In the present study, we propose a bio-inspired tactile system with a population of 23 digital afferents: 12 RA-I, 6 SA-I, and 5 nociceptors. Indeed, the functional concept of the nociceptor is implemented on the FPGA for the first time. One of the main features of biological tactile afferents is that their distal axon branches in the skin, creating complex receptive fields. Given these physiological observations, the bio-inspired afferents are randomly connected to the several neighboring mechanoreceptors with different weights to form their own receptive field. To test the performance of the proposed neuromorphic chip in sharpness detection, a robotic system with three-degree of freedom equipped with the tactile sensor indents the 3D-printed objects. Spike responses of the biomimetic afferents are then collected for analysis by rate and temporal coding algorithms. In this way, the impact of the innervation mechanism and collaboration of afferents and nociceptors on sharpness recognition are investigated. Our findings suggest that the synergy between sensory afferents and nociceptors conveys more information about tactile stimuli which in turn leads to the robustness of the proposed neuromorphic system against damage to the taxels or afferents. Moreover, it is illustrated that spiking activity of the biomimetic nociceptors is amplified as the sharpness increases which can be considered as a feedback mechanism for prosthesis protection. This neuromorphic approach advances the development of prosthesis to include the sensory feedback and to distinguish innocuous (non-painful) and noxious (painful) stimuli.
RESUMEN
Cerebellar synaptic plasticity is vital for adaptability and fine tuning of goal-directed movements. The perceived sensory errors between desired and actual movement outcomes are commonly considered to induce plasticity in the cerebellar synapses, with an objective to improve desirability of the executed movements. In rapid goal-directed eye movements called saccades, the only available sensory feedback is the direction of reaching error information received only at end of the movement. Moreover, this sensory error dependent plasticity can only improve the accuracy of the movements, while ignoring other essential characteristics such as reaching in minimum-time. In this work we propose a rate based, cerebellum inspired adaptive filter model to address refinement of both accuracy and movement-time of saccades. We use optimal control approach in conjunction with information constraints posed by the cerebellum to derive bio-plausible supervised plasticity rules. We implement and validate this bio-inspired scheme on a humanoid robot. We found out that, separate plasticity mechanisms in the model cerebellum separately control accuracy and movement-time. These plasticity mechanisms ensure that optimal saccades are produced by just receiving the direction of end reaching error as an evaluative signal. Furthermore, the model emulates encoding in the cerebellum of movement kinematics as observed in biological experiments.
Asunto(s)
Cerebelo , Movimientos Sacádicos , Fenómenos Biomecánicos , MovimientoRESUMEN
Being able to replicate real experiments with computational simulations is a unique opportunity to refine and validate models with experimental data and redesign the experiments based on simulations. However, since it is technically demanding to model all components of an experiment, traditional approaches to modeling reduce the experimental setups as much as possible. In this study, our goal is to replicate all the relevant features of an experiment on motor control and motor rehabilitation after stroke. To this aim, we propose an approach that allows continuous integration of new experimental data into a computational modeling framework. First, results show that we could reproduce experimental object displacement with high accuracy via the simulated embodiment in the virtual world by feeding a spinal cord model with experimental registration of the cortical activity. Second, by using computational models of multiple granularities, our preliminary results show the possibility of simulating several features of the brain after stroke, from the local alteration in neuronal activity to long-range connectivity remodeling. Finally, strategies are proposed to merge the two pipelines. We further suggest that additional models could be integrated into the framework thanks to the versatility of the proposed approach, thus allowing many researchers to achieve continuously improved experimental design.
RESUMEN
The cerebellum, which is responsible for motor control and learning, has been suggested to act as a Smith predictor for compensation of time-delays by means of internal forward models. However, insights about how forward model predictions are integrated in the Smith predictor have not yet been unveiled. To fill this gap, a novel bio-inspired modular control architecture that merges a recurrent cerebellar-like loop for adaptive control and a Smith predictor controller is proposed. The goal is to provide accurate anticipatory corrections to the generation of the motor commands in spite of sensory delays and to validate the robustness of the proposed control method to input and physical dynamic changes. The outcome of the proposed architecture with other two control schemes that do not include the Smith control strategy or the cerebellar-like corrections are compared. The results obtained on four sets of experiments confirm that the cerebellum-like circuit provides more effective corrections when only the Smith strategy is adopted and that minor tuning in the parameters, fast adaptation and reproducible configuration are enabled.
Asunto(s)
Adaptación Fisiológica/fisiología , Anticipación Psicológica/fisiología , Cerebelo/fisiología , Aprendizaje/fisiología , Modelos Biológicos , Actividad Motora/fisiología , HumanosRESUMEN
One of the big challenges in robotics is to endow agents with autonomous and adaptive capabilities. With this purpose, we embedded a cerebellum-based control system into a humanoid robot that becomes capable of handling dynamical external and internal complexity. The cerebellum is the area of the brain that coordinates and predicts the body movements throughout the body-environment interactions. Different biologically plausible cerebellar models are available in literature and have been employed for motor learning and control of simplified objects. We built the canonical cerebellar microcircuit by combining machine learning and computational neuroscience techniques. The control system is composed of the adaptive cerebellar module and a classic control method; their combination allows a fast adaptive learning and robust control of the robotic movements when external disturbances appear. The control structure is built offline, but the dynamic parameters are learned during an online-phase training. The aforementioned adaptive control system has been tested in the Neuro-robotics Platform with the virtual humanoid robot iCub. In the experiment, the robot iCub has to balance with the hand a table with a ball running on it. In contrast with previous attempts of solving this task, the proposed neural controller resulted able to quickly adapt when the internal and external conditions change. Our bio-inspired and flexible control architecture can be applied to different robotic configurations without an excessive tuning of the parameters or customization. The cerebellum-based control system is indeed able to deal with changing dynamics and interactions with the environment. Important insights regarding the relationship between the bio-inspired control system functioning and the complexity of the task to be performed are obtained.
RESUMEN
In traditional robotics, model-based controllers are usually needed in order to bring a robotic plant to the next desired state, but they present critical issues when the dimensionality of the control problem increases and disturbances from the external environment affect the system behavior, in particular during locomotion tasks. It is generally accepted that the motion control of quadruped animals is performed by neural circuits located in the spinal cord that act as a Central Pattern Generator and can generate appropriate locomotion patterns. This is thought to be the result of evolutionary processes that have optimized this network. On top of this, fine motor control is learned during the lifetime of the animal thanks to the plastic connections of the cerebellum that provide descending corrective inputs. This research aims at understanding and identifying the possible advantages of using learning during an evolution-inspired optimization for finding the best locomotion patterns in a robotic locomotion task. Accordingly, we propose a comparative study between two bio-inspired control architectures for quadruped legged robots where learning takes place either during the evolutionary search or only after that. The evolutionary process is carried out in a simulated environment, on a quadruped legged robot. To verify the possibility of overcoming the reality gap, the performance of both systems has been analyzed by changing the robot dynamics and its interaction with the external environment. Results show better performance metrics for the robotic agent whose locomotion method has been discovered by applying the adaptive module during the evolutionary exploration for the locomotion trajectories. Even when the motion dynamics and the interaction with the environment is altered, the locomotion patterns found on the learning robotic system are more stable, both in the joint and in the task space.
RESUMEN
Traditionally, human vision research has focused on specific paradigms and proposed models to explain very specific properties of visual perception. However, the complexity and scope of modern psychophysical paradigms undermine the success of this approach. For example, perception of an element strongly deteriorates when neighboring elements are presented in addition (visual crowding). As it was shown recently, the magnitude of deterioration depends not only on the directly neighboring elements but on almost all elements and their specific configuration. Hence, to fully explain human visual perception, one needs to take large parts of the visual field into account and combine all the aspects of vision that become relevant at such scale. These efforts require sophisticated and collaborative modeling. The Neurorobotics Platform (NRP) of the Human Brain Project offers a unique opportunity to connect models of all sorts of visual functions, even those developed by different research groups, into a coherently functioning system. Here, we describe how we used the NRP to connect and simulate a segmentation model, a retina model, and a saliency model to explain complex results about visual perception. The combination of models highlights the versatility of the NRP and provides novel explanations for inward-outward anisotropy in visual crowding.