SafeRPlan: Safe deep reinforcement learning for intraoperative planning of pedicle screw placement.

Spinal fusion surgery requires highly accurate implantation of pedicle screw implants, which must be conducted in critical proximity to vital structures with a limited view of the anatomy. Robotic surgery systems have been proposed to improve placement accuracy. Despite remarkable advances, current robotic systems still lack advanced mechanisms for continuous updating of surgical plans during procedures, which hinders attaining higher levels of robotic autonomy. These systems adhere to conventional rigid registration concepts, relying on the alignment of preoperative planning to the intraoperative anatomy. In this paper, we propose a safe deep reinforcement learning (DRL) planning approach (SafeRPlan) for robotic spine surgery that leverages intraoperative observation for continuous path planning of pedicle screw placement. The main contributions of our method are (1) the capability to ensure safe actions by introducing an uncertainty-aware distance-based safety filter; (2) the ability to compensate for incomplete intraoperative anatomical information, by encoding a-priori knowledge of anatomical structures with neural networks pre-trained on pre-operative images; and (3) the capability to generalize over unseen observation noise thanks to the novel domain randomization techniques. Planning quality was assessed by quantitative comparison with the baseline approaches, gold standard (GS) and qualitative evaluation by expert surgeons. In experiments with human model datasets, our approach was capable of achieving over 5% higher safety rates compared to baseline approaches, even under realistic observation noise. To the best of our knowledge, SafeRPlan is the first safety-aware DRL planning approach specifically designed for robotic spine surgery.

Population-level coding of avoidance learning in medial prefrontal cortex.

The medial prefrontal cortex (mPFC) has been proposed to link sensory inputs and behavioral outputs to mediate the execution of learned behaviors. However, how such a link is implemented has remained unclear. To measure prefrontal neural correlates of sensory stimuli and learned behaviors, we performed population calcium imaging during a new tone-signaled active avoidance paradigm in mice. We developed an analysis approach based on dimensionality reduction and decoding that allowed us to identify interpretable task-related population activity patterns. While a large fraction of tone-evoked activity was not informative about behavior execution, we identified an activity pattern that was predictive of tone-induced avoidance actions and did not occur for spontaneous actions with similar motion kinematics. Moreover, this avoidance-specific activity differed between distinct avoidance actions learned in two consecutive tasks. Overall, our results are consistent with a model in which mPFC contributes to the selection of goal-directed actions by transforming sensory inputs into specific behavioral outputs through distributed population-level computations.

Directed and acyclic synaptic connectivity in the human layer 2-3 cortical microcircuit.

The computational capabilities of neuronal networks are fundamentally constrained by their specific connectivity. Previous studies of cortical connectivity have mostly been carried out in rodents; whether the principles established therein also apply to the evolutionarily expanded human cortex is unclear. We studied network properties within the human temporal cortex using samples obtained from brain surgery. We analyzed multineuron patch-clamp recordings in layer 2-3 pyramidal neurons and identified substantial differences compared with rodents. Reciprocity showed random distribution, synaptic strength was independent from connection probability, and connectivity of the supragranular temporal cortex followed a directed and mostly acyclic graph topology. Application of these principles in neuronal models increased dimensionality of network dynamics, suggesting a critical role for cortical computation.

Bio-inspired, task-free continual learning through activity regularization.

The ability to sequentially learn multiple tasks without forgetting is a key skill of biological brains, whereas it represents a major challenge to the field of deep learning. To avoid catastrophic forgetting, various continual learning (CL) approaches have been devised. However, these usually require discrete task boundaries. This requirement seems biologically implausible and often limits the application of CL methods in the real world where tasks are not always well defined. Here, we take inspiration from neuroscience, where sparse, non-overlapping neuronal representations have been suggested to prevent catastrophic forgetting. As in the brain, we argue that these sparse representations should be chosen on the basis of feed forward (stimulus-specific) as well as top-down (context-specific) information. To implement such selective sparsity, we use a bio-plausible form of hierarchical credit assignment known as Deep Feedback Control (DFC) and combine it with a winner-take-all sparsity mechanism. In addition to sparsity, we introduce lateral recurrent connections within each layer to further protect previously learned representations. We evaluate the new sparse-recurrent version of DFC on the split-MNIST computer vision benchmark and show that only the combination of sparsity and intra-layer recurrent connections improves CL performance with respect to standard backpropagation. Our method achieves similar performance to well-known CL methods, such as Elastic Weight Consolidation and Synaptic Intelligence, without requiring information about task boundaries. Overall, we showcase the idea of adopting computational principles from the brain to derive new, task-free learning algorithms for CL.

AI to the rescue of voltage imaging.

In a recent issue of Nature Methods, Platisa et al. present an approach for long-term, in vivo population voltage imaging with single spike resolution across a local population of 100 neurons.1 Key to this step forward was the combination of a customized high-speed two-photon microscope with an optimized, positive-going, genetically encoded voltage indicator and a tailored machine learning denoising algorithm.

Learning cortical hierarchies with temporal Hebbian updates.

A key driver of mammalian intelligence is the ability to represent incoming sensory information across multiple abstraction levels. For example, in the visual ventral stream, incoming signals are first represented as low-level edge filters and then transformed into high-level object representations. Similar hierarchical structures routinely emerge in artificial neural networks (ANNs) trained for object recognition tasks, suggesting that similar structures may underlie biological neural networks. However, the classical ANN training algorithm, backpropagation, is considered biologically implausible, and thus alternative biologically plausible training methods have been developed such as Equilibrium Propagation, Deep Feedback Control, Supervised Predictive Coding, and Dendritic Error Backpropagation. Several of those models propose that local errors are calculated for each neuron by comparing apical and somatic activities. Notwithstanding, from a neuroscience perspective, it is not clear how a neuron could compare compartmental signals. Here, we propose a solution to this problem in that we let the apical feedback signal change the postsynaptic firing rate and combine this with a differential Hebbian update, a rate-based version of classical spiking time-dependent plasticity (STDP). We prove that weight updates of this form minimize two alternative loss functions that we prove to be equivalent to the error-based losses used in machine learning: the inference latency and the amount of top-down feedback necessary. Moreover, we show that the use of differential Hebbian updates works similarly well in other feedback-based deep learning frameworks such as Predictive Coding or Equilibrium Propagation. Finally, our work removes a key requirement of biologically plausible models for deep learning and proposes a learning mechanism that would explain how temporal Hebbian learning rules can implement supervised hierarchical learning.

Different encoding of reward location in dorsal and intermediate hippocampus.

Hippocampal place cells fire at specific locations in the environment. They form a cognitive map that encodes spatial relations in the environment, including reward locations.1 As part of this encoding, dorsal CA1 (dCA1) place cells accumulate at reward.2-5 The encoding of learned reward location could vary between the dorsal and intermediate hippocampus, which differ in gene expression and cortical and subcortical connectivity.6 While the dorsal hippocampus is critical for spatial navigation, the involvement of intermediate CA1 (iCA1) in spatial navigation might depend on task complexity7 and learning phase.8-10 The intermediate-to-ventral hippocampus regulates reward-seeking,11-15 but little is known about the involvement in reward-directed navigation. Here, we compared the encoding of learned reward locations in dCA1 and iCA1 during spatial navigation. We used calcium imaging with a head-mounted microscope to track the activity of CA1 cells over multiple days during which mice learned different reward locations. In dCA1, the fraction of active place cells increased in anticipation of reward, but the pool of active cells changed with the reward location. In iCA1, the same cells anticipated multiple reward locations. Our results support a model in which the dCA1 cognitive map incorporates a changing population of cells that encodes reward proximity through increased population activity, while iCA1 provides a reward-predictive code through a dedicated subpopulation. Both of these location-invariant codes persisted over time, and together they provide a dual hippocampal reward location code, assisting goal-directed navigation.16,17.

Fast Retrograde Access to Projection Neuron Circuits Underlying Vocal Learning in Songbirds.

Understanding the structure and function of neural circuits underlying speech and language is a vital step toward better treatments for diseases of these systems. Songbirds, among the few animal orders that share with humans the ability to learn vocalizations from a conspecific, have provided many insights into the neural mechanisms of vocal development. However, research into vocal learning circuits has been hindered by a lack of tools for rapid genetic targeting of specific neuron populations to meet the quick pace of developmental learning. Here, we present a viral tool that enables fast and efficient retrograde access to projection neuron populations. In zebra finches, Bengalese finches, canaries, and mice, we demonstrate fast retrograde labeling of cortical or dopaminergic neurons. We further demonstrate the suitability of our construct for detailed morphological analysis, for in vivo imaging of calcium activity, and for multi-color brainbow labeling.

Wide. Fast. Deep: Recent Advances in Multiphoton Microscopy of In Vivo Neuronal Activity.

Multiphoton microscopy (MPM) has emerged as one of the most powerful and widespread technologies to monitor the activity of neuronal networks in awake, behaving animals over long periods of time. MPM development spanned across decades and crucially depended on the concurrent improvement of calcium indicators that report neuronal activity as well as surgical protocols, head fixation approaches, and innovations in optics and microscopy technology. Here we review the last decade of MPM development and highlight how in vivo imaging has matured and diversified, making it now possible to concurrently monitor thousands of neurons across connected brain areas or, alternatively, small local networks with sampling rates in the kilohertz range. This review includes different laser scanning approaches, such as multibeam technologies as well as recent developments to image deeper into neuronal tissues using new, long-wavelength laser sources. As future development will critically depend on our ability to resolve and discriminate individual neuronal spikes, we will also describe a simple framework that allows performing quantitative comparisons between the reviewed MPM instruments. Finally, we provide our own opinion on how the most recent MPM developments can be leveraged at scale to enable the next generation of discoveries in brain function.

Amygdala ensembles encode behavioral states.

Internal states, including affective or homeostatic states, are important behavioral motivators. The amygdala regulates motivated behaviors, yet how distinct states are represented in amygdala circuits is unknown. By longitudinally imaging neural calcium dynamics in freely moving mice across different environments, we identified opponent changes in activity levels of two major, nonoverlapping populations of basal amygdala principal neurons. This population signature does not report global anxiety but predicts switches between exploratory and nonexploratory, defensive states. Moreover, the amygdala separately processes external stimuli and internal states and broadcasts state information via several output pathways to larger brain networks. Our findings extend the concept of thalamocortical "brain-state" coding to include affective and exploratory states and provide an entry point into the state dependency of brain function and behavior in defined circuits.

An amygdalar neural ensemble that encodes the unpleasantness of pain.

Pain is an unpleasant experience. How the brain's affective neural circuits attribute this aversive quality to nociceptive information remains unknown. By means of time-lapse in vivo calcium imaging and neural activity manipulation in freely behaving mice encountering noxious stimuli, we identified a distinct neural ensemble in the basolateral amygdala that encodes the negative affective valence of pain. Silencing this nociceptive ensemble alleviated pain affective-motivational behaviors without altering the detection of noxious stimuli, withdrawal reflexes, anxiety, or reward. Following peripheral nerve injury, innocuous stimuli activated this nociceptive ensemble to drive dysfunctional perceptual changes associated with neuropathic pain, including pain aversion to light touch (allodynia). These results identify the amygdalar representations of noxious stimuli that are functionally required for the negative affective qualities of acute and chronic pain perception.

Diametric neural ensemble dynamics in parkinsonian and dyskinetic states.

Loss of dopamine in Parkinson's disease is hypothesized to impede movement by inducing hypo- and hyperactivity in striatal spiny projection neurons (SPNs) of the direct (dSPNs) and indirect (iSPNs) pathways in the basal ganglia, respectively. The opposite imbalance might underlie hyperkinetic abnormalities, such as dyskinesia caused by treatment of Parkinson's disease with the dopamine precursor L-DOPA. Here we monitored thousands of SPNs in behaving mice, before and after dopamine depletion and during L-DOPA-induced dyskinesia. Normally, intermingled clusters of dSPNs and iSPNs coactivated before movement. Dopamine depletion unbalanced SPN activity rates and disrupted the movement-encoding iSPN clusters. Matching their clinical efficacy, L-DOPA or agonism of the D2 dopamine receptor reversed these abnormalities more effectively than agonism of the D1 dopamine receptor. The opposite pathophysiology arose in L-DOPA-induced dyskinesia, during which iSPNs showed hypoactivity and dSPNs showed unclustered hyperactivity. Therefore, both the spatiotemporal profiles and rates of SPN activity appear crucial to striatal function, and next-generation treatments for basal ganglia disorders should target both facets of striatal activity.

Neuronal Representation of Social Information in the Medial Amygdala of Awake Behaving Mice.

The medial amygdala (MeA) plays a critical role in processing species- and sex-specific signals that trigger social and defensive behaviors. However, the principles by which this deep brain structure encodes social information is poorly understood. We used a miniature microscope to image the Ca2+ dynamics of large neural ensembles in awake behaving mice and tracked the responses of MeA neurons over several months. These recordings revealed spatially intermingled subsets of MeA neurons with distinct temporal dynamics. The encoding of social information in the MeA differed between males and females and relied on information from both individual cells and neuronal populations. By performing long-term Ca2+ imaging across different social contexts, we found that sexual experience triggers lasting and sex-specific changes in MeA activity, which, in males, involve signaling by oxytocin. These findings reveal basic principles underlying the brain's representation of social information and its modulation by intrinsic and extrinsic factors.

Social behaviour shapes hypothalamic neural ensemble representations of conspecific sex.

All animals possess a repertoire of innate (or instinctive) behaviours, which can be performed without training. Whether such behaviours are mediated by anatomically distinct and/or genetically specified neural pathways remains unknown. Here we report that neural representations within the mouse hypothalamus, that underlie innate social behaviours, are shaped by social experience. Oestrogen receptor 1-expressing (Esr1+) neurons in the ventrolateral subdivision of the ventromedial hypothalamus (VMHvl) control mating and fighting in rodents. We used microendoscopy to image Esr1+ neuronal activity in the VMHvl of male mice engaged in these social behaviours. In sexually and socially experienced adult males, divergent and characteristic neural ensembles represented male versus female conspecifics. However, in inexperienced adult males, male and female intruders activated overlapping neuronal populations. Sex-specific neuronal ensembles gradually separated as the mice acquired social and sexual experience. In mice permitted to investigate but not to mount or attack conspecifics, ensemble divergence did not occur. However, 30 minutes of sexual experience with a female was sufficient to promote the separation of male and female ensembles and to induce an attack response 24 h later. These observations uncover an unexpected social experience-dependent component to the formation of hypothalamic neural assemblies controlling innate social behaviours. More generally, they reveal plasticity and dynamic coding in an evolutionarily ancient deep subcortical structure that is traditionally viewed as a 'hard-wired' system.

Neural ensemble dynamics underlying a long-term associative memory.

The brain's ability to associate different stimuli is vital for long-term memory, but how neural ensembles encode associative memories is unknown. Here we studied how cell ensembles in the basal and lateral amygdala encode associations between conditioned and unconditioned stimuli (CS and US, respectively). Using a miniature fluorescence microscope, we tracked the Ca2+ dynamics of ensembles of amygdalar neurons during fear learning and extinction over 6 days in behaving mice. Fear conditioning induced both up- and down-regulation of individual cells' CS-evoked responses. This bi-directional plasticity mainly occurred after conditioning, and reshaped the neural ensemble representation of the CS to become more similar to the US representation. During extinction training with repetitive CS presentations, the CS representation became more distinctive without reverting to its original form. Throughout the experiments, the strength of the ensemble-encoded CS-US association predicted the level of behavioural conditioning in each mouse. These findings support a supervised learning model in which activation of the US representation guides the transformation of the CS representation.

Distinct Hippocampal Pathways Mediate Dissociable Roles of Context in Memory Retrieval.

Memories about sensory experiences are tightly linked to the context in which they were formed. Memory contextualization is fundamental for the selection of appropriate behavioral reactions needed for survival, yet the underlying neuronal circuits are poorly understood. By combining trans-synaptic viral tracing and optogenetic manipulation, we found that the ventral hippocampus (vHC) and the amygdala, two key brain structures encoding context and emotional experiences, interact via multiple parallel pathways. A projection from the vHC to the basal amygdala mediates fear behavior elicited by a conditioned context, whereas a parallel projection from a distinct subset of vHC neurons onto midbrain-projecting neurons in the central amygdala is necessary for context-dependent retrieval of cued fear memories. Our findings demonstrate that two fundamentally distinct roles of context in fear memory retrieval are processed by distinct vHC output pathways, thereby allowing for the formation of robust contextual fear memories while preserving context-dependent behavioral flexibility.

Reticulon 4A/Nogo-A influences the distribution of Kir4.1 but is not essential for potassium conductance in retinal Müller glia.

In the adult retina, we have previously shown that Nogo-A was highly expressed in Müller glia. However, the role of Nogo-A in the glial cell physiology is not clear. In this study, we investigated the possible influence that Nogo-A may exert on other polarized molecules in Müller cells, in particular inwardly rectifying potassium channel 4.1 (Kir4.1) and aquaporin 4 (AQP4) that respectively control potassium and water exchange in glial cells. Our results showed that adenovirus-mediated Nogo-A overexpression with AdNogo-A increased the immunofluorescent signal of Kir4.1 in rat Müller cell line 1 (rMC-1) cells but did not change its expression level by Western blotting. In vivo, AdNogo-A induced ectopic Kir4.1 immunoreactivity throughout the radial processes of Müller cells compared with AdLacZ control virus. Surprisingly, AdNogo-A did not modify the distribution of Dp71 and AQP4 that are common binding partners for Kir4.1 in the dystrophin-associated protein (DAP) complex anchored at the plasma membrane of Müller glia. Immunoprecipitation experiments revealed molecular interactions between Nogo-A and Kir4.1. In Nogo-A KO mouse retinae, the distribution of Kir4.1 was not different from that observed in Wild-Type (WT) animals. In addition, potassium conductance did not change in freshly dissociated Nogo-A KO Müller glia compared with WT cells. In summary, the increase of Nogo-A expression can selectively influence the distribution of Kir4.1 in glia but is not essential for Kir4.1-mediated potassium conductance at the plasma membrane in physiological conditions. Nogo-A-Kir4.1 interactions may, however, contribute to pathological processes taking place in the retina, for instance, after ischemia.

High-speed recording of neural spikes in awake mice and flies with a fluorescent voltage sensor.

Genetically encoded voltage indicators (GEVIs) are a promising technology for fluorescence readout of millisecond-scale neuronal dynamics. Previous GEVIs had insufficient signaling speed and dynamic range to resolve action potentials in live animals. We coupled fast voltage-sensing domains from a rhodopsin protein to bright fluorophores through resonance energy transfer. The resulting GEVIs are sufficiently bright and fast to report neuronal action potentials and membrane voltage dynamics in awake mice and flies, resolving fast spike trains with 0.2-millisecond timing precision at spike detection error rates orders of magnitude better than previous GEVIs. In vivo imaging revealed sensory-evoked responses, including somatic spiking, dendritic dynamics, and intracellular voltage propagation. These results empower in vivo optical studies of neuronal electrophysiology and coding and motivate further advancements in high-speed microscopy.

Cellular level brain imaging in behaving mammals: an engineering approach.

Fluorescence imaging offers expanding capabilities for recording neural dynamics in behaving mammals, including the means to monitor hundreds of cells targeted by genetic type or connectivity, track cells over weeks, densely sample neurons within local microcircuits, study cells too inactive to isolate in extracellular electrical recordings, and visualize activity in dendrites, axons, or dendritic spines. We discuss recent progress and future directions for imaging in behaving mammals from a systems engineering perspective, which seeks holistic consideration of fluorescent indicators, optical instrumentation, and computational analyses. Today, genetically encoded indicators of neural Ca(2+) dynamics are widely used, and those of trans-membrane voltage are rapidly improving. Two complementary imaging paradigms involve conventional microscopes for studying head-restrained animals and head-mounted miniature microscopes for imaging in freely behaving animals. Overall, the field has attained sufficient sophistication that increased cooperation between those designing new indicators, light sources, microscopes, and computational analyses would greatly benefit future progress.