Deep kinematic inference affords efficient and scalable control of bodily movements.

Performing goal-directed movements requires mapping goals from extrinsic (workspace-relative) to intrinsic (body-relative) coordinates and then to motor signals. Mainstream approaches based on optimal control realize the mappings by minimizing cost functions, which is computationally demanding. Instead, active inference uses generative models to produce sensory predictions, which allows a cheaper inversion to the motor signals. However, devising generative models to control complex kinematic chains like the human body is challenging. We introduce an active inference architecture that affords a simple but effective mapping from extrinsic to intrinsic coordinates via inference and easily scales up to drive complex kinematic chains. Rich goals can be specified in both intrinsic and extrinsic coordinates using attractive or repulsive forces. The proposed model reproduces sophisticated bodily movements and paves the way for computationally efficient and biologically plausible control of actuated systems.

Model-based spatial navigation in the hippocampus-ventral striatum circuit: A computational analysis.

While the neurobiology of simple and habitual choices is relatively well known, our current understanding of goal-directed choices and planning in the brain is still limited. Theoretical work suggests that goal-directed computations can be productively associated to model-based (reinforcement learning) computations, yet a detailed mapping between computational processes and neuronal circuits remains to be fully established. Here we report a computational analysis that aligns Bayesian nonparametrics and model-based reinforcement learning (MB-RL) to the functioning of the hippocampus (HC) and the ventral striatum (vStr)-a neuronal circuit that increasingly recognized to be an appropriate model system to understand goal-directed (spatial) decisions and planning mechanisms in the brain. We test the MB-RL agent in a contextual conditioning task that depends on intact hippocampus and ventral striatal (shell) function and show that it solves the task while showing key behavioral and neuronal signatures of the HC-vStr circuit. Our simulations also explore the benefits of biological forms of look-ahead prediction (forward sweeps) during both learning and control. This article thus contributes to fill the gap between our current understanding of computational algorithms and biological realizations of (model-based) reinforcement learning.

Computational foundations of the visual number sense.

We provide an emergentist perspective on the computational mechanism underlying numerosity perception, its development, and the role of inhibition, based on our deep neural network model. We argue that the influence of continuous visual properties does not challenge the notion of number sense, but reveals limit conditions for the computation that yields invariance in numerosity perception. Alternative accounts should be formalized in a computational model.

Prefrontal Goal Codes Emerge as Latent States in Probabilistic Value Learning.

The prefrontal cortex (PFC) supports goal-directed actions and exerts cognitive control over behavior, but the underlying coding and mechanism are heavily debated. We present evidence for the role of goal coding in PFC from two converging perspectives: computational modeling and neuronal-level analysis of monkey data. We show that neural representations of prospective goals emerge by combining a categorization process that extracts relevant behavioral abstractions from the input data and a reward-driven process that selects candidate categories depending on their adaptive value; both forms of learning have a plausible neural implementation in PFC. Our analyses demonstrate a fundamental principle: goal coding represents an efficient solution to cognitive control problems, analogous to efficient coding principles in other (e.g., visual) brain areas. The novel analytical-computational approach is of general interest because it applies to a variety of neurophysiological studies.

Functional organization of the insula and inner perisylvian regions.

In the last few years, the insula has been the focus of many brain-imaging studies, mostly devoted to clarify its role in emotions and social communication. Physiological data, however, on which one may ground these correlative findings are almost totally lacking. Here, we investigated the functional properties of the insular cortex in behaving monkeys using intracortical microstimulation. Behavioral responses and heart rate changes were recorded. The results showed that the insula is functionally formed by two main subdivisions: (i) a sensorimotor field occupying the caudal-dorsal portion of the insula and appearing as an extension of the parietal lobe; and (ii) a mosaic of orofacial motor programs located in the anterior and centroventral insula sector. These programs show a progressive shift from dorsally located nonemotional motor programs (ingestive activity) to ventral ones laden with emotional and communicative content. The relationship between ingestive and other behaviors is discussed in an evolutionary perspective.

Number skills are maintained in healthy ageing.

Numerical skills have been extensively studied in terms of their development and pathological decline, but whether they change in healthy ageing is not well known. Longer exposure to numbers and quantity-related problems may progressively refine numerical skills, similar to what happens to other cognitive abilities like verbal memory. Alternatively, number skills may be sensitive to ageing, reflecting either a decline of number processing itself or of more auxiliary cognitive abilities that are involved in number tasks. To distinguish between these possibilities we tested 30 older and 30 younger participants on an established numerosity discrimination task requiring to judge which of two sets of items is more numerous, and on arithmetical tasks. Older participants were remarkably accurate in performing arithmetical tasks although their numerosity discrimination (also known as 'number acuity') was impaired. Further analyses indicate that this impairment was limited to numerosity trials requiring inhibiting information incongruent to numerosity (e.g., fewer but larger items), and that this also correlated with poor inhibitory processes measured by standard tests. Therefore, rather than a numerical impairment, poor numerosity discrimination is likely to reflect elderly's impoverished inhibitory processes. This conclusion is supported by simulations with a recent neuro-computational model of numerosity perception, where only the specific degradation of inhibitory processes produced a pattern that closely resembled older participants' performance. Numeracy seems therefore resilient to ageing but it is influenced by the decline of inhibitory processes supporting number performance, consistent with the 'Inhibitory Deficit' Theory.

Neural representation in active inference: Using generative models to interact with-and understand-the lived world.

This paper considers neural representation through the lens of active inference, a normative framework for understanding brain function. It delves into how living organisms employ generative models to minimize the discrepancy between predictions and observations (as scored with variational free energy). The ensuing analysis suggests that the brain learns generative models to navigate the world adaptively, not (or not solely) to understand it. Different living organisms may possess an array of generative models, spanning from those that support action-perception cycles to those that underwrite planning and imagination; namely, from explicit models that entail variables for predicting concurrent sensations, like objects, faces, or people-to action-oriented models that predict action outcomes. It then elucidates how generative models and belief dynamics might link to neural representation and the implications of different types of generative models for understanding an agent's cognitive capabilities in relation to its ecological niche. The paper concludes with open questions regarding the evolution of generative models and the development of advanced cognitive abilities-and the gradual transition from pragmatic to detached neural representations. The analysis on offer foregrounds the diverse roles that generative models play in cognitive processes and the evolution of neural representation.

Flexible intentions: An Active Inference theory.

We present a normative computational theory of how the brain may support visually-guided goal-directed actions in dynamically changing environments. It extends the Active Inference theory of cortical processing according to which the brain maintains beliefs over the environmental state, and motor control signals try to fulfill the corresponding sensory predictions. We propose that the neural circuitry in the Posterior Parietal Cortex (PPC) compute flexible intentions-or motor plans from a belief over targets-to dynamically generate goal-directed actions, and we develop a computational formalization of this process. A proof-of-concept agent embodying visual and proprioceptive sensors and an actuated upper limb was tested on target-reaching tasks. The agent behaved correctly under various conditions, including static and dynamic targets, different sensory feedbacks, sensory precisions, intention gains, and movement policies; limit conditions were individuated, too. Active Inference driven by dynamic and flexible intentions can thus support goal-directed behavior in constantly changing environments, and the PPC might putatively host its core intention mechanism. More broadly, the study provides a normative computational basis for research on goal-directed behavior in end-to-end settings and further advances mechanistic theories of active biological systems.

Active Vision in Binocular Depth Estimation: A Top-Down Perspective.

Depth estimation is an ill-posed problem; objects of different shapes or dimensions, even if at different distances, may project to the same image on the retina. Our brain uses several cues for depth estimation, including monocular cues such as motion parallax and binocular cues such as diplopia. However, it remains unclear how the computations required for depth estimation are implemented in biologically plausible ways. State-of-the-art approaches to depth estimation based on deep neural networks implicitly describe the brain as a hierarchical feature detector. Instead, in this paper we propose an alternative approach that casts depth estimation as a problem of active inference. We show that depth can be inferred by inverting a hierarchical generative model that simultaneously predicts the eyes' projections from a 2D belief over an object. Model inversion consists of a series of biologically plausible homogeneous transformations based on Predictive Coding principles. Under the plausible assumption of a nonuniform fovea resolution, depth estimation favors an active vision strategy that fixates the object with the eyes, rendering the depth belief more accurate. This strategy is not realized by first fixating on a target and then estimating the depth; instead, it combines the two processes through action-perception cycles, with a similar mechanism of the saccades during object recognition. The proposed approach requires only local (top-down and bottom-up) message passing, which can be implemented in biologically plausible neural circuits.

The hippocampal formation as a hierarchical generative model supporting generative replay and continual learning.

We advance a novel computational theory of the hippocampal formation as a hierarchical generative model that organizes sequential experiences, such as rodent trajectories during spatial navigation, into coherent spatiotemporal contexts. We propose that the hippocampal generative model is endowed with inductive biases to identify individual items of experience (first hierarchical layer), organize them into sequences (second layer) and cluster them into maps (third layer). This theory entails a novel characterization of hippocampal reactivations as generative replay: the offline resampling of fictive sequences from the generative model, which supports the continual learning of multiple sequential experiences. We show that the model learns and efficiently retains multiple spatial navigation trajectories, by organizing them into spatial maps. Furthermore, the model reproduces flexible and prospective aspects of hippocampal dynamics that are challenging to explain within existing frameworks. This theory reconciles multiple roles of the hippocampal formation in map-based navigation, episodic memory and imagination.

The dynamics of reading complex words: evidence from steady-state visual evoked potentials.

The present study used steady-state visual evoked potentials (SSVEPs) to examine the spatio-temporal dynamics of reading morphologically complex words and test the neurophysiological activation pattern elicited by stems and suffixes. Three different types of target words were presented to proficient readers in a delayed naming task: truly suffixed words (e.g., farmer), pseudo-suffixed words (e.g., corner), and non-suffixed words (e.g., cashew). Embedded stems and affixes were flickered at two different frequencies (18.75 Hz and 12.50 Hz, respectively). The stem data revealed an earlier SSVEP peak in the truly suffixed and pseudo-suffixed conditions compared to the non-suffixed condition, thus providing evidence for the form-based activation of embedded stems during reading. The suffix data also showed a dissociation in the SSVEP response between suffixes and non-suffixes with an additional activation boost for truly suffixed words. The observed differences are discussed in the context of current models of complex word recognition.

Steady state visual evoked potentials in reading aloud: Effects of lexicality, frequency and orthographic familiarity.

The present study explored the possibility to use Steady-State Visual Evoked Potentials (SSVEPs) as a tool to investigate the core mechanisms in visual word recognition. In particular, we investigated three benchmark effects of reading aloud: lexicality (words vs. pseudowords), frequency (high-frequency vs. low-frequency words), and orthographic familiarity ('familiar' versus 'unfamiliar' pseudowords). We found that words and pseudowords elicited robust SSVEPs. Words showed larger SSVEPs than pseudowords and high-frequency words showed larger SSVEPs than low-frequency words. SSVEPs were not sensitive to orthographic familiarity. We further localized the neural generators of the SSVEP effects. The lexicality effect was located in areas associated with early level of visual processing, i.e. in the right occipital lobe and in the right precuneus. Pseudowords produced more activation than words in left sensorimotor areas, rolandic operculum, insula, supramarginal gyrus and in the right temporal gyrus. These areas are devoted to speech processing and/or spelling-to-sound conversion. The frequency effect involved the left temporal pole and orbitofrontal cortex, areas previously implicated in semantic processing and stimulus-response associations respectively, and the right postcentral and parietal inferior gyri, possibly indicating the involvement of the right attentional network.

Visuospatial priming of the mental number line.

It has been argued that numbers are spatially organized along a "mental number line" that facilitates left-hand responses to small numbers, and right-hand responses to large numbers. We hypothesized that whenever the representations of visual and numerical space are concurrently activated, interactions can occur between them, before response selection. A spatial prime is processed faster than a numerical target, and consistent with our hypothesis, we found that such a spatial prime affects non-spatial, verbal responses more when the prime follows a numerical target (backward priming) then when it precedes it (forward priming). This finding emerged both in a number-comparison and a parity judgment task, and cannot be ascribed to a "Spatial-Numerical Association of Response Codes" (SNARC). Contrary to some earlier claims, we therefore conclude that visuospatial-numerical interactions do occur, even before response selection.

Neuroplus biofeedback improves attention, resilience, and injury prevention in elite soccer players.

Performance and injury prevention in elite soccer players are typically investigated from physical-tactical, biomechanical, and metabolic perspectives. However, executive functions, visuospatial abilities, and psychophysiological adaptability or resilience are also fundamental for efficiency and well-being in sports. Based on previous research associating autonomic flexibility with prefrontal cortical control, we designed a novel integrated autonomic biofeedback training method called Neuroplus to improve resilience, visual attention, and injury prevention. Herein, we introduce the method and provide an evaluation of 20 elite soccer players from the Italian Soccer High Division (Serie-A): 10 players trained with Neuroplus and 10 trained with a control treatment. The assessments included psychophysiological stress profiles, a visual search task, and indexes of injury prevention, which were measured pre- and posttreatment. The analysis showed a significant enhancement of physiological adaptability, recovery following stress, visual selective attention, and injury prevention that were specific to the Neuroplus group. Enhancing the interplay between autonomic and cognitive functions through biofeedback may become a key principle for obtaining excellence and well-being in sports. To our knowledge, this is the first evidence that shows improvement in visual selective attention following intense autonomic biofeedback.

Letter perception emerges from unsupervised deep learning and recycling of natural image features.

The use of written symbols is a major achievement of human cultural evolution. However, how abstract letter representations might be learned from vision is still an unsolved problem 1,2 . Here, we present a large-scale computational model of letter recognition based on deep neural networks 3,4 , which develops a hierarchy of increasingly more complex internal representations in a completely unsupervised way by fitting a probabilistic, generative model to the visual input 5,6 . In line with the hypothesis that learning written symbols partially recycles pre-existing neuronal circuits for object recognition 7 , earlier processing levels in the model exploit domain-general visual features learned from natural images, while domain-specific features emerge in upstream neurons following exposure to printed letters. We show that these high-level representations can be easily mapped to letter identities even for noise-degraded images, producing accurate simulations of a broad range of empirical findings on letter perception in human observers. Our model shows that by reusing natural visual primitives, learning written symbols only requires limited, domain-specific tuning, supporting the hypothesis that their shape has been culturally selected to match the statistical structure of natural environments 8 .

Learning Orthographic Structure With Sequential Generative Neural Networks.

Learning the structure of event sequences is a ubiquitous problem in cognition and particularly in language. One possible solution is to learn a probabilistic generative model of sequences that allows making predictions about upcoming events. Though appealing from a neurobiological standpoint, this approach is typically not pursued in connectionist modeling. Here, we investigated a sequential version of the restricted Boltzmann machine (RBM), a stochastic recurrent neural network that extracts high-order structure from sensory data through unsupervised generative learning and can encode contextual information in the form of internal, distributed representations. We assessed whether this type of network can extract the orthographic structure of English monosyllables by learning a generative model of the letter sequences forming a word training corpus. We show that the network learned an accurate probabilistic model of English graphotactics, which can be used to make predictions about the letter following a given context as well as to autonomously generate high-quality pseudowords. The model was compared to an extended version of simple recurrent networks, augmented with a stochastic process that allows autonomous generation of sequences, and to non-connectionist probabilistic models (n-grams and hidden Markov models). We conclude that sequential RBMs and stochastic simple recurrent networks are promising candidates for modeling cognition in the temporal domain.

Modeling language and cognition with deep unsupervised learning: a tutorial overview.

Deep unsupervised learning in stochastic recurrent neural networks with many layers of hidden units is a recent breakthrough in neural computation research. These networks build a hierarchy of progressively more complex distributed representations of the sensory data by fitting a hierarchical generative model. In this article we discuss the theoretical foundations of this approach and we review key issues related to training, testing and analysis of deep networks for modeling language and cognitive processing. The classic letter and word perception problem of McClelland and Rumelhart (1981) is used as a tutorial example to illustrate how structured and abstract representations may emerge from deep generative learning. We argue that the focus on deep architectures and generative (rather than discriminative) learning represents a crucial step forward for the connectionist modeling enterprise, because it offers a more plausible model of cortical learning as well as a way to bridge the gap between emergentist connectionist models and structured Bayesian models of cognition.

Deep Unsupervised Learning on a Desktop PC: A Primer for Cognitive Scientists.

Deep belief networks hold great promise for the simulation of human cognition because they show how structured and abstract representations may emerge from probabilistic unsupervised learning. These networks build a hierarchy of progressively more complex distributed representations of the sensory data by fitting a hierarchical generative model. However, learning in deep networks typically requires big datasets and it can involve millions of connection weights, which implies that simulations on standard computers are unfeasible. Developing realistic, medium-to-large-scale learning models of cognition would therefore seem to require expertise in programing parallel-computing hardware, and this might explain why the use of this promising approach is still largely confined to the machine learning community. Here we show how simulations of deep unsupervised learning can be easily performed on a desktop PC by exploiting the processors of low cost graphic cards (graphic processor units) without any specific programing effort, thanks to the use of high-level programming routines (available in MATLAB or Python). We also show that even an entry-level graphic card can outperform a small high-performance computing cluster in terms of learning time and with no loss of learning quality. We therefore conclude that graphic card implementations pave the way for a widespread use of deep learning among cognitive scientists for modeling cognition and behavior.

Emergence of a 'visual number sense' in hierarchical generative models.

Numerosity estimation is phylogenetically ancient and foundational to human mathematical learning, but its computational bases remain controversial. Here we show that visual numerosity emerges as a statistical property of images in 'deep networks' that learn a hierarchical generative model of the sensory input. Emergent numerosity detectors had response profiles resembling those of monkey parietal neurons and supported numerosity estimation with the same behavioral signature shown by humans and animals.

Interactions between perceptual and numerical space.

Interactions between numbers and space have become a major issue in numerical cognition. Neuropsychological studies suggest that the interactions occur, before response selection, at a spatially organized representation of numbers (the mental number line). Reaction time (RT) studies, on the other hand, usually point to associations between response codes that do not necessarily imply a number line. There is only one such study that has found a spationumerical interaction between perception and semantics (SNIPS) effect before response selection. Here, in Experiment 1, we isolated the SNIPS effect from other numerical effects and corroborated the prediction that it can be induced by both left and right spatial cues. In Experiment 2, we isolated the peak of the time course of the SNIPS effect and corroborated the prediction that it occurs when a cue follows a target, and not when both appear simultaneously. The results reconcile neuropsychological and RT studies and support the hypothesis that numbers are represented along a left-to-right spatially organized mental number line. [corrected]