Meta-Learning Strategies through Value Maximization in Neural Networks.

Biological and artificial learning agents face numerous choices about how to learn, ranging from hyperparameter selection to aspects of task distributions like curricula. Understanding how to make these meta-learning choices could offer normative accounts of cognitive control functions in biological learners and improve engineered systems. Yet optimal strategies remain challenging to compute in modern deep networks due to the complexity of optimizing through the entire learning process. Here we theoretically investigate optimal strategies in a tractable setting. We present a learning effort framework capable of efficiently optimizing control signals on a fully normative objective: discounted cumulative performance throughout learning. We obtain computational tractability by using average dynamical equations for gradient descent, available for simple neural network architectures. Our framework accommodates a range of meta-learning and automatic curriculum learning methods in a unified normative setting. We apply this framework to investigate the effect of approximations in common meta-learning algorithms; infer aspects of optimal curricula; and compute optimal neuronal resource allocation in a continual learning setting. Across settings, we find that control effort is most beneficial when applied to easier aspects of a task early in learning; followed by sustained effort on harder aspects. Overall, the learning effort framework provides a tractable theoretical test bed to study normative benefits of interventions in a variety of learning systems, as well as a formal account of optimal cognitive control strategies over learning trajectories posited by established theories in cognitive neuroscience.

Pushing the Bounds of Bounded Optimality and Rationality.

All forms of cognition, whether natural or artificial, are subject to constraints of their computing architecture. This assumption forms the tenet of virtually all general theories of cognition, including those deriving from bounded optimality and bounded rationality. In this letter, we highlight an unresolved puzzle related to this premise: what are these constraints, and why are cognitive architectures subject to cognitive constraints in the first place? First, we lay out some pieces along the puzzle edge, such as computational tradeoffs inherent to neural architectures that give rise to rational bounds of cognition. We then outline critical next steps for characterizing cognitive bounds, proposing that some of these bounds can be subject to modification by cognition and, as such, are part of what is being optimized when cognitive agents decide how to allocate cognitive resources. We conclude that these emerging views may contribute to a more holistic perspective on the nature of cognitive bounds, as well as their alteration subject to cognition.

Strategically managing learning during perceptual decision making.

Making optimal decisions in the face of noise requires balancing short-term speed and accuracy. But a theory of optimality should account for the fact that short-term speed can influence long-term accuracy through learning. Here, we demonstrate that long-term learning is an important dynamical dimension of the speed-accuracy trade-off. We study learning trajectories in rats and formally characterize these dynamics in a theory expressed as both a recurrent neural network and an analytical extension of the drift-diffusion model that learns over time. The model reveals that choosing suboptimal response times to learn faster sacrifices immediate reward, but can lead to greater total reward. We empirically verify predictions of the theory, including a relationship between stimulus exposure and learning speed, and a modulation of reaction time by future learning prospects. We find that rats' strategies approximately maximize total reward over the full learning epoch, suggesting cognitive control over the learning process.

Encoding of 3D Head Orienting Movements in the Primary Visual Cortex.

Animals actively sample the sensory world by generating complex patterns of movement that evolve in three dimensions. Whether or how such movements affect neuronal activity in sensory cortical areas remains largely unknown, because most experiments exploring movement-related modulation have been performed in head-fixed animals. Here, we show that 3D head-orienting movements (HOMs) modulate primary visual cortex (V1) activity in a direction-specific manner that also depends on light. We identify two overlapping populations of movement-direction-tuned neurons that support this modulation, one of which is direction tuned in the dark and the other in the light. Although overall movement enhanced V1 responses to visual stimulation, HOMs suppressed responses. We demonstrate that V1 receives a motor efference copy related to orientation from secondary motor cortex, which is involved in controlling HOMs. These results support predictive coding theories of brain function and reveal a pervasive role of 3D movement in shaping sensory cortical dynamics.

A Micro-CT-based Method for Characterizing Lesions and Locating Electrodes in Small Animal Brains.

Lesion and electrode location verification are traditionally done via histological examination of stained brain slices, a time-consuming procedure that requires manual estimation. Here, we describe a simple, straightforward method for quantifying lesions and locating electrodes in the brain that is less laborious and yields more detailed results. Whole brains are stained with osmium tetroxide, embedded in resin, and imaged with a micro-CT scanner. The scans result in 3D digital volumes of the brains with resolutions and virtual section thicknesses dependent on the sample size (12-15 and 5-6 µm per voxel for rat and zebra finch brains, respectively). Surface and deep lesions can be characterized, and single tetrodes, tetrode arrays, electrolytic lesions, and silicon probes can also be localized. Free and proprietary software allows experimenters to examine the sample volume from any plane and segment the volume manually or automatically. Because this method generates whole brain volume, lesions and electrodes can be quantified to a much higher degree than in current methods, which will help standardize comparisons within and across studies.

A micro-CT-based method for quantitative brain lesion characterization and electrode localization.

Lesion verification and quantification is traditionally done via histological examination of sectioned brains, a time-consuming process that relies heavily on manual estimation. Such methods are particularly problematic in posterior cortical regions (e.g. visual cortex), where sectioning leads to significant damage and distortion of tissue. Even more challenging is the post hoc localization of micro-electrodes, which relies on the same techniques, suffers from similar drawbacks and requires even higher precision. Here, we propose a new, simple method for quantitative lesion characterization and electrode localization that is less labor-intensive and yields more detailed results than conventional methods. We leverage staining techniques standard in electron microscopy with the use of commodity micro-CT imaging. We stain whole rat and zebra finch brains in osmium tetroxide, embed these in resin and scan entire brains in a micro-CT machine. The scans result in 3D reconstructions of the brains with section thickness dependent on sample size (12-15 and 5-6 microns for rat and zebra finch respectively) that can be segmented manually or automatically. Because the method captures the entire intact brain volume, comparisons within and across studies are more tractable, and the extent of lesions and electrodes may be studied with higher accuracy than with current methods.